-- Logs begin at Fri 2022-09-09 15:52:10 UTC, end at Fri 2022-09-09 16:53:41 UTC. -- Sep 09 16:16:55.124310 dv-cinder-831247-27-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 09 16:16:55.133281 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 09 16:16:55.135303 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 9 16:16:55 2022] *** Sep 09 16:16:55.135303 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 09 16:16:55.135303 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 09 16:16:55.135303 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: nodename: dv-cinder-831247-27-iscsi Sep 09 16:16:55.135303 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: machine: x86_64 Sep 09 16:16:55.135303 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: clock source: unix Sep 09 16:16:55.135303 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: pcre jit disabled Sep 09 16:16:55.135303 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: detected number of CPU cores: 4 Sep 09 16:16:55.135303 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: current working directory: / Sep 09 16:16:55.135303 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: detected binary path: /usr/bin/uwsgi-core Sep 09 16:16:55.135303 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: your processes number limit is 47680 Sep 09 16:16:55.135876 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: your memory page size is 4096 bytes Sep 09 16:16:55.135876 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: detected max file descriptor number: 2048 Sep 09 16:16:55.135876 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: lock engine: pthread robust mutexes Sep 09 16:16:55.135876 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: thunder lock: enabled Sep 09 16:16:55.135876 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 09 16:16:55.135876 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 09 16:16:55.173918 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: Python main interpreter initialized at 0x559a13d819c0 Sep 09 16:16:55.173918 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: python threads support enabled Sep 09 16:16:55.173918 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: your server socket listen backlog is limited to 100 connections Sep 09 16:16:55.173918 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: your mercy for graceful operations on workers is 90 seconds Sep 09 16:16:55.174260 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: mapped 403077 bytes (393 KB) for 2 cores Sep 09 16:16:55.174329 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: *** Operational MODE: preforking *** Sep 09 16:16:55.174614 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: *** uWSGI is running in multiple interpreter mode *** Sep 09 16:16:55.174614 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: spawned uWSGI master process (pid: 81498) Sep 09 16:16:55.174814 dv-cinder-831247-27-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 09 16:16:55.182548 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: spawned uWSGI worker 1 (pid: 81510, cores: 1) Sep 09 16:16:55.182548 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: spawned uWSGI worker 2 (pid: 81511, cores: 1) Sep 09 16:16:55.182548 dv-cinder-831247-27-iscsi devstack@keystone.service[81498]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 09 16:16:59.368335 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 16:16:59.368335 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: from cryptography.utils import int_from_bytes Sep 09 16:16:59.369141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 16:16:59.369141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: from cryptography.utils import int_from_bytes Sep 09 16:16:59.369141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 16:16:59.369141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: from cryptography.utils import int_from_bytes Sep 09 16:16:59.369630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 16:16:59.369630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: from cryptography.utils import int_from_bytes Sep 09 16:16:59.418054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 16:16:59.418252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 16:16:59.418337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 16:16:59.418412 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 16:16:59.418471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] command line args: [] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 16:16:59.418570 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] command line args: [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 16:16:59.418711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 16:16:59.418841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 16:16:59.418917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ================================================================================ {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 16:16:59.418987 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ================================================================================ {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 16:16:59.419160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] admin_token = **** {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.419258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] admin_token = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.419333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] config_dir = [] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.419452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] config_dir = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.419533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.419711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.419846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] config_source = [] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.419934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] config_source = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.419999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] debug = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.420062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] debug = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.420119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.420366 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.420461 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.420526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.420588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.420644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.420774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.420859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.421055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.421166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.421240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] list_limit = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.421299 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] list_limit = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.421427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] log_config_append = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.421513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] log_config_append = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.421578 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.421639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.421754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] log_dir = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.421826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] log_dir = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.421916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] log_file = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.421994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] log_file = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.422172 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.422283 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.422392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.422557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.422648 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.422727 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.422910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.423026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.423101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.423171 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.423249 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.423322 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.423461 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.423558 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.423733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.423868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.423955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.424016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.424146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.424223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.424415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.424557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.424650 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.424723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.424832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.424955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.425135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.425263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.425351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.425423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.425853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.425967 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.426048 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] publish_errors = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.426108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] publish_errors = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.426263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.426380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.426457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.426539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.426654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.426726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.426871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.426967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.427115 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.427236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.427349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] standard_threads = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.427435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] standard_threads = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.427607 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.427736 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.427804 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.427863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.427948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.428128 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.428128 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] use_journal = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.428388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] use_journal = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.428460 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] use_json = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.428533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] use_json = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.428626 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] use_stderr = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.428695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] use_stderr = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.428773 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] use_syslog = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.428842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] use_syslog = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.428914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.428981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 16:16:59.429045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.429117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.429273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.429358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.429589 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.429675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.429828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.429904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.430076 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.430152 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.430309 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.430389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.430909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.430987 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.431143 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.431219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.431386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.431466 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.431629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.431708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.431877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.431956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.432105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.432184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.432347 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.432426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.432577 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.432653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.432820 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.432899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.433062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.433136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.433295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.433374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.433546 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.433627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.433797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.433876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.434024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.434094 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.434256 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.434336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.434336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.434767 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.434767 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.434893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.435016 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.435090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.435262 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.435343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.435489 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.435601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.435717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.435790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.435941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.436026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.436173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.436247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.436419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.436491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.436662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.436747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.436892 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.436966 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.437124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.437196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.437367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.437448 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.437610 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.437683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.437842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.437921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.438088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.438168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.438314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.438385 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.438567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.438655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.438804 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.438888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.439032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.439106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.439259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.439332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.439486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.439640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.439711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.439778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.439932 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.440009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.440176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.440258 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.440404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.440477 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.440636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.440714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.440862 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.440947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.441094 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.441200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.441309 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.441402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.441554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.441667 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.441762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.441853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.442002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.442120 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.442604 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.442703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.442842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.442918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.443071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.443153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.443295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.443373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.443520 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.443631 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.443765 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.443836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.443998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.444075 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.444233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.444307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.444460 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.444582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.444716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.444839 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.444947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.445025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.445212 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.445296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.445446 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.445522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.445679 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.445758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.445932 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.446013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.446163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.446241 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.446406 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.446486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.446671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.446747 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.446904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.446984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.447135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.447211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.447367 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.447443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.447610 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.447684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.447843 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.447923 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.448075 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.448151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.448311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.448385 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.448544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.448624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.448786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.448861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.449016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.449090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.449246 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.449322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.449477 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.449551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.449707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.449793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.449936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.450012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.450190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.450264 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.450425 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.450553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.450692 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.450769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.450928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.451003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.451160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.451241 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.451389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.451461 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.451615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.451691 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.451843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.451926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.452072 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.452156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.452297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.452375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.452522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.452602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.452748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.452831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.452979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.453053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.453203 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.453277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.453439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.453523 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.453671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.453745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.453919 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.453994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.454149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.454231 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.454402 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.454477 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.454677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.454755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.454916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.455028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.455143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] auth.external = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.455227 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] auth.external = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.455370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.455445 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.455663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.455797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.455885 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.455964 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.456104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] auth.password = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.456181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] auth.password = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.456328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] auth.token = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.456403 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] auth.token = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.456561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.456645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.456792 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.456868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.457034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.457115 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.457261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.457345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.457504 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.457579 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.457743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.457818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.457968 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.458051 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.458199 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] credential.caching = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.458274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] credential.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.458441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.458562 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.458713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.458799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.458956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.459032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.459193 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.459267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.459429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.459516 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.459659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.459741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.459904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.459979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.460154 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.460237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.460386 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.460465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.460638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.460715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.460918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.461004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.461162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.461238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.461420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.461498 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.461657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.461741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.461910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.461986 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.462141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] federation.caching = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.462216 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.462373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.462457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.462641 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.462719 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.462885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.462961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.463113 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.463200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.463359 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.463436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.463590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.463665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.463840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.463917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.464071 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.464146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.464322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.464404 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.464553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.464630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.464796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.464871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.465027 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.465103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity.caching = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.465273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.465352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.465516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.465591 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.465745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.465861 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.465974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.466057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.466218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.466294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.466442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.466557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.466700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.466785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.466945 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.467020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.467188 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.467263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.467413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.467490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.467638 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.467713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.467864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.467940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.468102 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.468180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.468345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.468421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.468588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.468664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.468838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.468915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.469079 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.469153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.469348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.469433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.469580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.469657 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.469811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.469886 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.470037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.470111 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.470269 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.470345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.470492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.470586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.470744 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.470819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.470978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.471056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.471210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.471286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.471453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.471529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.471696 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.471771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.471910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.471987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.472150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.472225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.472380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.472458 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.472620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.472728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.472863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.472948 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.473088 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.473171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.473321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.473398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.473548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.473633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.473776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.473897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.474005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.474084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.474238 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.474323 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.474470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.474595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.474857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.474936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.475087 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.475176 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.475335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.475412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.475566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.475642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.475794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.475879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.476044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.476125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.476291 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.476367 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.476572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.476656 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.476805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.476880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.477040 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.477116 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.477266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.477349 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.477503 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.477581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.477759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.477835 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.477987 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.478070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.478232 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.478314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.478477 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.478573 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.478744 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.478832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.478976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.479053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.479203 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.479279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.479435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.479517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.479664 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.479741 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.479896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.479972 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.480122 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.480208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.480367 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.480442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.480608 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.480689 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.480850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.480934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.481091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.481167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.481333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.481714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.481714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.481845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.481916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.481986 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.482055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.482123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.482230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.482316 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.482460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.482590 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.482716 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.482792 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.482954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.483043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.483199 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.483275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.483434 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.483509 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.483687 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.483770 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.483915 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.483991 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.484152 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.484227 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.484382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.484465 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.484611 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.484687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.484843 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.484919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.485083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.485167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.485321 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.485396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.485547 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.485621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.485778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.485862 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.486013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.486089 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] resource.caching = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.486258 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.486333 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.486517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.486608 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.486755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.486832 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.487001 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.487077 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.487243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.487327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.487475 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.487552 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.487717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.487793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.487948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.488099 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.488174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.488243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.488401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] role.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.488478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] role.caching = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.488644 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] role.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.488723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] role.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.488868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.488944 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.489110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.489186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.489353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.489430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.489597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.489672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.489839 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.489914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.490080 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.490157 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.490321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.490396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.490585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.490663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.490832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.490911 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.491059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.491134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.491300 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.491375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.491542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.491619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.491784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.491859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.492053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.492128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.492438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.492516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.492670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.492745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.492919 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.492994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.493164 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.493241 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.493410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.493486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.493655 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.493731 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.493883 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.493960 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.494130 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.494208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.494376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.494460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.494635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.494711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.494862 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.494938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.495095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.495182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.495327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.495403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.495572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.495647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.495822 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.495905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.496064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.496139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.496294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.496369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.496524 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.496607 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.496749 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.496824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.496979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] token.caching = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.497053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.497209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.497292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.497454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.497530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.497684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.497759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.497936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.498019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.498181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.498256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.498413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.498487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.498675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.498766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.498916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.498993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.499163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.499239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.499396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.499480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.499631 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.499708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.499864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.499939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.500108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.500191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.500381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.500457 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.500607 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.500691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.500846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.500930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.501108 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.501189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.501363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.connection = **** {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.501440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.connection = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.501602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.501677 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.501847 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.501922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.502080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.502163 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.502316 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.502391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.502572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.502654 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.502806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.502884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.503036 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.503111 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.503268 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.503343 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.503511 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.503588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.503742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.503818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.503981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.504056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.504212 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.504289 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.504454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.504529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.504683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.504758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.504921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.504998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.505146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.505221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.505379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.505454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.505609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.505686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 16:16:59.505776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81510) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 16:16:59.505852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81511) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 16:16:59.534661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.534890 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.535146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.535270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.535595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.535595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.543463 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.543954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.571420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.571652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.571808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.572037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.623989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.624147 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.671583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.671800 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.672069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.672269 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.829602 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.829810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.829880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.830021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.830173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.830244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.830312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.830385 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.830445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.830659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.830783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.830850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.853470 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.853715 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.853818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.853921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.853999 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.854074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.854135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.854334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.854491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.854613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.854689 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.854761 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.854822 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.855017 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.865782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.866065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.878582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.878826 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.878938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.878938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.879097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.879169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.879229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.879425 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.879561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.879631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.879692 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.879764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81510) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.879825 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.880016 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81511) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 16:16:59.896540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.896771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 16:16:59.897091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.897213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.897282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 16:16:59.897366 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 16:16:59.897555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.897701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.897770 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.897831 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 16:16:59.898054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.898218 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.898371 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.898521 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.898688 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.898926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.907944 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.908223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.908447 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.908577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.908729 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.908872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.908944 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.909257 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.915749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.916223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.918679 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.918967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.919096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.919248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.919374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.919527 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.919676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.919766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.919918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.920044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.920194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.920333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.920404 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.920569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.920695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.920838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.920978 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.921089 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.921182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.921359 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.921482 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.921693 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.944428 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.944551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.946011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.946135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.946295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.946439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.950948 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.951070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.951217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.951341 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.951516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.951643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.957824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.957951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.958112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.958237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.963813 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.963942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.964103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.964230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.964387 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.964533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.964687 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.964813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.964926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.965019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.965232 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.965359 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.965467 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.965559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.965759 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.965885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.981087 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.981209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.981401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.981523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.985161 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.985304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.988379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.988504 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.988659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.988783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.992976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.993103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.996106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.996234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:16:59.999334 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.999478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.999606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.999698 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:16:59.999913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.000040 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.000158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.000249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.000460 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.000584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.000736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.000873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.016248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.016481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.016579 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.016777 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.016898 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.017016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.017104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.017309 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.023853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.024206 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.025359 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.025734 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.026878 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.027155 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.027299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.027545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.029755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.030037 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.030201 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.030298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.030528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.030676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.030825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.030949 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.031103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.031490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.042067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.042350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.042609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.042733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.042891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.043036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.043174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.043326 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.043418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.043649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.043786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.043932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.044057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.044366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.059226 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.059786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.062093 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.062697 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.065007 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.065564 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.066443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.067022 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.068277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.068548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.068849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.068972 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.069137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.069505 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.075347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.075937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.076799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.077403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.079717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.080111 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.080315 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.080559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.080706 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.080853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.080991 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.081233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.088473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.088776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.089053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.089257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.089415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.089699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.095905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81511) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.096189 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.096463 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.096585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81510) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 16:17:00.096750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.096881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.097007 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.097088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.097213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.097367 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.097515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.097674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.097815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.097908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.098017 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.098225 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.098351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.098458 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.098592 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.098771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.098895 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.099049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.099173 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.099314 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.099412 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81511) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.099574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.099875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.100151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81510) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 16:17:00.133293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x559a13d819c0 pid: 81511 (default app) Sep 09 16:17:00.134029 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x559a13d819c0 pid: 81510 (default app) Sep 09 16:17:00.136588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfbbc2-6ba9-4a99-bd66-f7ff5d95f422 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:00.136870 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfbbc2-6ba9-4a99-bd66-f7ff5d95f422 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:00.137142 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfbbc2-6ba9-4a99-bd66-f7ff5d95f422 None None] PATH_INFO: `/v3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:00.141367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Fri Sep 9 16:17:00 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:06.951404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13ffa7-926a-4a89-8622-335795550e8e None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:06.952212 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13ffa7-926a-4a89-8622-335795550e8e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:06.952212 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13ffa7-926a-4a89-8622-335795550e8e None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:06.959419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:06 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:06.966239 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c17100-25c1-40a0-ba94-ac779a4ad254 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:06.966730 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c17100-25c1-40a0-ba94-ac779a4ad254 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:06.966871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c17100-25c1-40a0-ba94-ac779a4ad254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:07.040785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: INFO dbcounter [None req-c0c17100-25c1-40a0-ba94-ac779a4ad254 None None] Registered counter for database keystone Sep 09 16:17:07.061730 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG oslo_db.sqlalchemy.engines [None req-c0c17100-25c1-40a0-ba94-ac779a4ad254 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81511) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 16:17:07.072761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writer thread running {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 16:17:07.261110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG passlib.handlers.bcrypt [None req-c0c17100-25c1-40a0-ba94-ac779a4ad254 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81511) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 16:17:07.261404 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG passlib.handlers.bcrypt [None req-c0c17100-25c1-40a0-ba94-ac779a4ad254 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81511) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 16:17:07.322581 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c0c17100-25c1-40a0-ba94-ac779a4ad254 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:07.324218 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c0c17100-25c1-40a0-ba94-ac779a4ad254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:07.464299 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 500 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 16:17:07.905620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13ffa7-926a-4a89-8622-335795550e8e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:07.978782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: INFO dbcounter [None req-5a13ffa7-926a-4a89-8622-335795550e8e None None] Registered counter for database keystone Sep 09 16:17:08.003578 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG oslo_db.sqlalchemy.engines [None req-5a13ffa7-926a-4a89-8622-335795550e8e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81510) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 16:17:08.013182 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writer thread running {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 16:17:08.134016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13ffa7-926a-4a89-8622-335795550e8e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:08.139538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879b2a3-fbaa-43e7-979b-a19fd613c7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:08.140181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879b2a3-fbaa-43e7-979b-a19fd613c7bd admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:08.140457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879b2a3-fbaa-43e7-979b-a19fd613c7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:08.140730 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879b2a3-fbaa-43e7-979b-a19fd613c7bd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:08.149696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879b2a3-fbaa-43e7-979b-a19fd613c7bd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:08.200779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9879b2a3-fbaa-43e7-979b-a19fd613c7bd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 16:17:08.201061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879b2a3-fbaa-43e7-979b-a19fd613c7bd admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:08.211436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Fri Sep 9 16:17:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:17:08.218463 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c17100-25c1-40a0-ba94-ac779a4ad254 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:08.245633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c17100-25c1-40a0-ba94-ac779a4ad254 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:08.251448 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6f0a6-4cc4-4392-bb37-cb1dc4c88483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:08.252582 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6f0a6-4cc4-4392-bb37-cb1dc4c88483 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:08.252852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6f0a6-4cc4-4392-bb37-cb1dc4c88483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:08.253120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6f0a6-4cc4-4392-bb37-cb1dc4c88483 admin admin] PATH_INFO: `/v3/projects/f5e628b418b44cb0b8fcf5460ded8ac8` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:08.254853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6f0a6-4cc4-4392-bb37-cb1dc4c88483 admin admin] RBAC: Authorizing `identity:get_project(project_id=f5e628b418b44cb0b8fcf5460ded8ac8)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:08.306452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fed6f0a6-4cc4-4392-bb37-cb1dc4c88483 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 16:17:08.306818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6f0a6-4cc4-4392-bb37-cb1dc4c88483 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:08.311402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Fri Sep 9 16:17:08 2022] GET /identity/v3/projects/f5e628b418b44cb0b8fcf5460ded8ac8 => generated 344 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:10.401241 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602aed1e-d197-4980-8606-27774e836cc6 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:10.401856 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602aed1e-d197-4980-8606-27774e836cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:10.401856 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602aed1e-d197-4980-8606-27774e836cc6 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:10.406009 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:17:10.412742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2575167-02c5-413a-a601-a7ab66e7caa5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:10.413472 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2575167-02c5-413a-a601-a7ab66e7caa5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:10.413630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2575167-02c5-413a-a601-a7ab66e7caa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:10.441392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c2575167-02c5-413a-a601-a7ab66e7caa5 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:10.443332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c2575167-02c5-413a-a601-a7ab66e7caa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:10.452756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 16:17:10.716034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602aed1e-d197-4980-8606-27774e836cc6 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:10.730683 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602aed1e-d197-4980-8606-27774e836cc6 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:10.736064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed134186-a7f6-4103-8d50-adea2806ff0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:10.736666 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed134186-a7f6-4103-8d50-adea2806ff0b admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:10.736946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed134186-a7f6-4103-8d50-adea2806ff0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:10.737215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed134186-a7f6-4103-8d50-adea2806ff0b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:10.745805 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed134186-a7f6-4103-8d50-adea2806ff0b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:10.747621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed134186-a7f6-4103-8d50-adea2806ff0b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:10.756960 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Fri Sep 9 16:17:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:17:10.762783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2575167-02c5-413a-a601-a7ab66e7caa5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:10.772896 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2575167-02c5-413a-a601-a7ab66e7caa5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:10.779202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838f61e-4666-4376-89c4-a21b0c01f874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:10.779835 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838f61e-4666-4376-89c4-a21b0c01f874 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:10.780106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838f61e-4666-4376-89c4-a21b0c01f874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:10.780370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838f61e-4666-4376-89c4-a21b0c01f874 admin admin] PATH_INFO: `/v3/users/94cbbd1503324af1af6df581aea570b9` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:10.781898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838f61e-4666-4376-89c4-a21b0c01f874 admin admin] RBAC: Authorizing `identity:get_user(user_id=94cbbd1503324af1af6df581aea570b9)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:10.783753 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838f61e-4666-4376-89c4-a21b0c01f874 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:10.785252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Fri Sep 9 16:17:10 2022] GET /identity/v3/users/94cbbd1503324af1af6df581aea570b9 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:12.980879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd89adb-a9bf-4f95-9d5b-2a31f65c9cfc None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:12.981451 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd89adb-a9bf-4f95-9d5b-2a31f65c9cfc None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:12.981451 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd89adb-a9bf-4f95-9d5b-2a31f65c9cfc None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:12.982786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:12.988178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f68b46-a2ea-46e2-afad-3496f5a491e8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:12.988626 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f68b46-a2ea-46e2-afad-3496f5a491e8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:12.988719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f68b46-a2ea-46e2-afad-3496f5a491e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:13.001039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa029502-a354-4233-b13f-6dd8c03c2c0e None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:13.001324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa029502-a354-4233-b13f-6dd8c03c2c0e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:13.001593 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa029502-a354-4233-b13f-6dd8c03c2c0e None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:13.002902 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:17:13.008261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406207e-c260-4124-bcb2-e12b8ef442e2 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:13.008437 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-42f68b46-a2ea-46e2-afad-3496f5a491e8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:13.008637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406207e-c260-4124-bcb2-e12b8ef442e2 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:13.008865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406207e-c260-4124-bcb2-e12b8ef442e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:13.009635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-42f68b46-a2ea-46e2-afad-3496f5a491e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:13.050776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG passlib.handlers.bcrypt [None req-b406207e-c260-4124-bcb2-e12b8ef442e2 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81510) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 16:17:13.051192 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG passlib.handlers.bcrypt [None req-b406207e-c260-4124-bcb2-e12b8ef442e2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81510) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 16:17:13.097299 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:13.099992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-b406207e-c260-4124-bcb2-e12b8ef442e2 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:13.101739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b406207e-c260-4124-bcb2-e12b8ef442e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:13.202468 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:13.360760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f68b46-a2ea-46e2-afad-3496f5a491e8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:13.386674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:13.467280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406207e-c260-4124-bcb2-e12b8ef442e2 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:13.467507 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f68b46-a2ea-46e2-afad-3496f5a491e8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:13.501391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdea9e1-5493-426d-866b-abb06e219049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:13.502025 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdea9e1-5493-426d-866b-abb06e219049 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:13.502305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdea9e1-5493-426d-866b-abb06e219049 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:13.502623 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdea9e1-5493-426d-866b-abb06e219049 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:13.513663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdea9e1-5493-426d-866b-abb06e219049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-4cdea9e1-5493-426d-866b-abb06e219049 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:13.522486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:13.523513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:13.524315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:13.524315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:13.524315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:13.524315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:13.524315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:13.524315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:13.524315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:13.524315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:13.524315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:13.524315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:13.524315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:13.524315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:13.524315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:17:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:13.529750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdea9e1-5493-426d-866b-abb06e219049 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:13.567803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406207e-c260-4124-bcb2-e12b8ef442e2 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:13.570125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:13.607085 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23b60f-2cfb-414e-af9f-86a8cc6a786a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:13.608304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23b60f-2cfb-414e-af9f-86a8cc6a786a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:13.608590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23b60f-2cfb-414e-af9f-86a8cc6a786a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:13.608855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23b60f-2cfb-414e-af9f-86a8cc6a786a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:13.612719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:13.630751 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdea9e1-5493-426d-866b-abb06e219049 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:13.665745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef273de8-71dc-4323-b9db-7e05e070b076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:13.666345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef273de8-71dc-4323-b9db-7e05e070b076 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:13.666658 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef273de8-71dc-4323-b9db-7e05e070b076 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:13.666932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef273de8-71dc-4323-b9db-7e05e070b076 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:13.667603 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef273de8-71dc-4323-b9db-7e05e070b076 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:13.667908 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef273de8-71dc-4323-b9db-7e05e070b076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:13.669320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef273de8-71dc-4323-b9db-7e05e070b076 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:13.680725 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23b60f-2cfb-414e-af9f-86a8cc6a786a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:13.685766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 8/15] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:13.685906 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23b60f-2cfb-414e-af9f-86a8cc6a786a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:13.686818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef273de8-71dc-4323-b9db-7e05e070b076 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:13.780654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 8/16] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 16:17:13.784789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef273de8-71dc-4323-b9db-7e05e070b076 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:13.786528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23b60f-2cfb-414e-af9f-86a8cc6a786a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:13.789260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:13.818034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8ad48-336c-48e4-a931-f351e95501a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:13.818790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8ad48-336c-48e4-a931-f351e95501a9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:13.819070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8ad48-336c-48e4-a931-f351e95501a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:13.819345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8ad48-336c-48e4-a931-f351e95501a9 None admin] PATH_INFO: `/v3/users/94cbbd1503324af1af6df581aea570b9` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:13.820553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8ad48-336c-48e4-a931-f351e95501a9 None admin] RBAC: Authorizing `identity:get_user(user_id=94cbbd1503324af1af6df581aea570b9)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:13.822019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8ad48-336c-48e4-a931-f351e95501a9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:13.823799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:13 2022] GET /identity/v3/users/94cbbd1503324af1af6df581aea570b9 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:13.829042 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8ad48-336c-48e4-a931-f351e95501a9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:13.843754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:13.884706 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23b60f-2cfb-414e-af9f-86a8cc6a786a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:13.918265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118a18c-7392-4a2d-9eed-5001dae1f2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:13.919177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118a18c-7392-4a2d-9eed-5001dae1f2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:13.919457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118a18c-7392-4a2d-9eed-5001dae1f2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:13.919724 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118a18c-7392-4a2d-9eed-5001dae1f2f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:13.926690 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8ad48-336c-48e4-a931-f351e95501a9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:13.938384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118a18c-7392-4a2d-9eed-5001dae1f2f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:13.939914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118a18c-7392-4a2d-9eed-5001dae1f2f0 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-5118a18c-7392-4a2d-9eed-5001dae1f2f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:17:13.959190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:13.960057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:17:13.960881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:17:13.960881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:17:13.960881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:13.961031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842fb90-14d7-455d-b7ed-71ac32a617d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:13.961121 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:13.961185 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842fb90-14d7-455d-b7ed-71ac32a617d9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:13.961185 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842fb90-14d7-455d-b7ed-71ac32a617d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:13.961294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842fb90-14d7-455d-b7ed-71ac32a617d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:13.965834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e842fb90-14d7-455d-b7ed-71ac32a617d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:13.966317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118a18c-7392-4a2d-9eed-5001dae1f2f0 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:13.967344 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e842fb90-14d7-455d-b7ed-71ac32a617d9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:13.968879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:13.974717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842fb90-14d7-455d-b7ed-71ac32a617d9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:14.022302 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:14.054773 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:14.068008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118a18c-7392-4a2d-9eed-5001dae1f2f0 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:14.074153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842fb90-14d7-455d-b7ed-71ac32a617d9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:14.107677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21da504-8bef-425c-a9e2-632700392a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:14.108297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21da504-8bef-425c-a9e2-632700392a1d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:14.108578 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21da504-8bef-425c-a9e2-632700392a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:14.108770 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564f3e2-d68a-4d7c-8356-5a81e0332ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:14.108870 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21da504-8bef-425c-a9e2-632700392a1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:14.109392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564f3e2-d68a-4d7c-8356-5a81e0332ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:14.109624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21da504-8bef-425c-a9e2-632700392a1d None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.domain.id=default, user.id=94cbbd1503324af1af6df581aea570b9) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:14.109717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564f3e2-d68a-4d7c-8356-5a81e0332ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:14.109933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564f3e2-d68a-4d7c-8356-5a81e0332ba6 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:14.110013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21da504-8bef-425c-a9e2-632700392a1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:14.110613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564f3e2-d68a-4d7c-8356-5a81e0332ba6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:14.110916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564f3e2-d68a-4d7c-8356-5a81e0332ba6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:14.111430 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21da504-8bef-425c-a9e2-632700392a1d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:14.112299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564f3e2-d68a-4d7c-8356-5a81e0332ba6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:14.118695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 11/20] 10.222.0.23 () {56 vars in 1322 bytes} [Fri Sep 9 16:17:13 2022] GET /identity/v3/role_assignments?user.id=94cbbd1503324af1af6df581aea570b9&scope.domain.id=default&role.id=c6da7ba2be504943af29338ee8855f05 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:14.135337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 10/21] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:16.560560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43befab-52cb-47a0-9e83-7bf8426eb699 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:16.561083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43befab-52cb-47a0-9e83-7bf8426eb699 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:16.561477 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43befab-52cb-47a0-9e83-7bf8426eb699 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:16.563097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 12/22] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:16.568852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff6ce9-33f7-4d86-a415-6aa6fcb0e497 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:16.569269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff6ce9-33f7-4d86-a415-6aa6fcb0e497 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:16.569436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff6ce9-33f7-4d86-a415-6aa6fcb0e497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:16.595954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-06ff6ce9-33f7-4d86-a415-6aa6fcb0e497 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:16.597509 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-06ff6ce9-33f7-4d86-a415-6aa6fcb0e497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:16.687789 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 11/23] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:17:16.979621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef51d1b-ffa5-4ebc-8608-6d571c2598df None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:16.980162 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef51d1b-ffa5-4ebc-8608-6d571c2598df None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:16.980593 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef51d1b-ffa5-4ebc-8608-6d571c2598df None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:16.982386 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 13/24] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:16.987937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381c7bf-0055-4d26-8b18-0ab0d2a96c02 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:16.988426 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381c7bf-0055-4d26-8b18-0ab0d2a96c02 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:16.988845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381c7bf-0055-4d26-8b18-0ab0d2a96c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:16.998604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c34d6-4fa9-492f-93ef-96cfd58665f1 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:16.999069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c34d6-4fa9-492f-93ef-96cfd58665f1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:16.999469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c34d6-4fa9-492f-93ef-96cfd58665f1 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:17.001252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 14/25] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:17.006256 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f986bbb-c458-452d-a8a3-5b4617aa3485 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:17.006731 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f986bbb-c458-452d-a8a3-5b4617aa3485 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:17.007158 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f986bbb-c458-452d-a8a3-5b4617aa3485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:17.028387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-8381c7bf-0055-4d26-8b18-0ab0d2a96c02 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:17.029919 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8381c7bf-0055-4d26-8b18-0ab0d2a96c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:17.044422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7f986bbb-c458-452d-a8a3-5b4617aa3485 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:17.045832 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7f986bbb-c458-452d-a8a3-5b4617aa3485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:17.109701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:17.141184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 12/26] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:17.160676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 15/27] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:17:17.256894 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381c7bf-0055-4d26-8b18-0ab0d2a96c02 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:17.381772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381c7bf-0055-4d26-8b18-0ab0d2a96c02 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:17.425621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59170efe-0ff3-4fc2-979a-3964f8b0f35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:17.426680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59170efe-0ff3-4fc2-979a-3964f8b0f35e None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:17.427177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59170efe-0ff3-4fc2-979a-3964f8b0f35e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:17.427657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59170efe-0ff3-4fc2-979a-3964f8b0f35e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:17.439001 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59170efe-0ff3-4fc2-979a-3964f8b0f35e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:17.441157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59170efe-0ff3-4fc2-979a-3964f8b0f35e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-59170efe-0ff3-4fc2-979a-3964f8b0f35e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:17:17.461065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:17.461985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:17:17.463054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:17:17.463054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:17:17.463054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:17.464950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 13/28] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:17.471343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f986bbb-c458-452d-a8a3-5b4617aa3485 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:17.500610 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58d8c0-5e98-45c4-bb68-b8431d4c95fe None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:17.501269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58d8c0-5e98-45c4-bb68-b8431d4c95fe None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:17.501735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58d8c0-5e98-45c4-bb68-b8431d4c95fe None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:17.503436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 14/29] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:17:17.511662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bc1c2-484a-4aef-bb93-39ecc86f9e14 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:17.512214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bc1c2-484a-4aef-bb93-39ecc86f9e14 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:17.512695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bc1c2-484a-4aef-bb93-39ecc86f9e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:17.531960 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-ca9bc1c2-484a-4aef-bb93-39ecc86f9e14 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:17.533605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ca9bc1c2-484a-4aef-bb93-39ecc86f9e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:17.576950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:17.589524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:17.632237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f986bbb-c458-452d-a8a3-5b4617aa3485 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:17.650657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 15/30] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:17:17.685314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46725faa-e094-4eb0-aebe-9fe258a24e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:17.686295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46725faa-e094-4eb0-aebe-9fe258a24e23 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:17.686804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46725faa-e094-4eb0-aebe-9fe258a24e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:17.687268 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46725faa-e094-4eb0-aebe-9fe258a24e23 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:17.688720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46725faa-e094-4eb0-aebe-9fe258a24e23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:17.689264 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46725faa-e094-4eb0-aebe-9fe258a24e23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:17.691093 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46725faa-e094-4eb0-aebe-9fe258a24e23 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:17.698731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9bc1c2-484a-4aef-bb93-39ecc86f9e14 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:17.737850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 16/31] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:17.739629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46725faa-e094-4eb0-aebe-9fe258a24e23 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:17.871220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9bc1c2-484a-4aef-bb93-39ecc86f9e14 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:17.912197 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46725faa-e094-4eb0-aebe-9fe258a24e23 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:17.943786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c6bde-c34a-48f7-8ddd-f649256ccfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:17.944448 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c6bde-c34a-48f7-8ddd-f649256ccfc3 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:17.944737 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c6bde-c34a-48f7-8ddd-f649256ccfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:17.945098 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c6bde-c34a-48f7-8ddd-f649256ccfc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:17.948401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261574a0-ea9e-4de0-985b-75e1ac5cb654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:17.949242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261574a0-ea9e-4de0-985b-75e1ac5cb654 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:17.949732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261574a0-ea9e-4de0-985b-75e1ac5cb654 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:17.950195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261574a0-ea9e-4de0-985b-75e1ac5cb654 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:17.951079 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261574a0-ea9e-4de0-985b-75e1ac5cb654 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:17.952766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261574a0-ea9e-4de0-985b-75e1ac5cb654 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:17.956936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:17.967949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c6bde-c34a-48f7-8ddd-f649256ccfc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-1e0c6bde-c34a-48f7-8ddd-f649256ccfc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:17.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:17.981769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:17.981769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:17.981769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:17.981769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:17.981769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:17.981769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:17.981769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:17.981769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:17.981769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:17.981769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:17.981769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:17.981769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:17.983930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 16/32] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:17:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:17:17.984929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c6bde-c34a-48f7-8ddd-f649256ccfc3 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:18.008179 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:18.092105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c6bde-c34a-48f7-8ddd-f649256ccfc3 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:18.092420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 17/33] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Sep 9 16:17:17 2022] POST /identity/v3/roles => generated 219 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:17:18.093520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261574a0-ea9e-4de0-985b-75e1ac5cb654 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:18.132026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bd0c1-07df-4434-94d4-6025c3b32c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:18.133012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bd0c1-07df-4434-94d4-6025c3b32c82 None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:18.133470 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bd0c1-07df-4434-94d4-6025c3b32c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:18.133915 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bd0c1-07df-4434-94d4-6025c3b32c82 None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:18.135115 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bd0c1-07df-4434-94d4-6025c3b32c82 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:18.136798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bd0c1-07df-4434-94d4-6025c3b32c82 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:18.295080 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 17/34] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Sep 9 16:17:17 2022] POST /identity/v3/projects => generated 301 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:17:18.301531 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bd0c1-07df-4434-94d4-6025c3b32c82 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:18.327015 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261574a0-ea9e-4de0-985b-75e1ac5cb654 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:18.410810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:18.425362 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441933d-0bd3-4e58-8899-78e9111dbf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:18.426171 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441933d-0bd3-4e58-8899-78e9111dbf9e None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:18.426718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441933d-0bd3-4e58-8899-78e9111dbf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:18.427203 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441933d-0bd3-4e58-8899-78e9111dbf9e None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:18.428110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441933d-0bd3-4e58-8899-78e9111dbf9e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:18.430745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441933d-0bd3-4e58-8899-78e9111dbf9e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:18.453866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bd0c1-07df-4434-94d4-6025c3b32c82 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:18.454088 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 18/35] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Sep 9 16:17:18 2022] POST /identity/v3/roles => generated 225 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:17:18.494727 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3244d-eedf-42e2-8595-31edbfb4c6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:18.495631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3244d-eedf-42e2-8595-31edbfb4c6fe None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:18.496056 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3244d-eedf-42e2-8595-31edbfb4c6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:18.496496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3244d-eedf-42e2-8595-31edbfb4c6fe None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:18.497442 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3244d-eedf-42e2-8595-31edbfb4c6fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:18.497926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3244d-eedf-42e2-8595-31edbfb4c6fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:18.499765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3244d-eedf-42e2-8595-31edbfb4c6fe None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:18.515817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 18/36] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:18.521931 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441933d-0bd3-4e58-8899-78e9111dbf9e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:18.627436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:18.661376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0132f5-c96d-4510-96d8-69b0bca778ca None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:18.662347 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0132f5-c96d-4510-96d8-69b0bca778ca None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:18.662853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0132f5-c96d-4510-96d8-69b0bca778ca None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:18.664636 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 19/37] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:17:18.670787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22becb-98e0-4217-b0d5-5374051d3877 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:18.671363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22becb-98e0-4217-b0d5-5374051d3877 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:18.671832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22becb-98e0-4217-b0d5-5374051d3877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:18.689486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441933d-0bd3-4e58-8899-78e9111dbf9e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:18.703417 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-1e22becb-98e0-4217-b0d5-5374051d3877 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:18.705120 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1e22becb-98e0-4217-b0d5-5374051d3877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:18.728115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de614cba-cd2b-4bda-b98d-96fd77aa5708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:18.728776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de614cba-cd2b-4bda-b98d-96fd77aa5708 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:18.729055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de614cba-cd2b-4bda-b98d-96fd77aa5708 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:18.729352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de614cba-cd2b-4bda-b98d-96fd77aa5708 None admin] PATH_INFO: `/v3/users/94cbbd1503324af1af6df581aea570b9` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:18.731150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de614cba-cd2b-4bda-b98d-96fd77aa5708 None admin] RBAC: Authorizing `identity:get_user(user_id=94cbbd1503324af1af6df581aea570b9)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:18.732739 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de614cba-cd2b-4bda-b98d-96fd77aa5708 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:18.734378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 19/38] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:18 2022] GET /identity/v3/users/94cbbd1503324af1af6df581aea570b9 => generated 247 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:18.735769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc11bce-3007-4d67-814f-9533805e9d84 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:18.736039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc11bce-3007-4d67-814f-9533805e9d84 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:18.736314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc11bce-3007-4d67-814f-9533805e9d84 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:18.737653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 20/39] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:18.741241 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc11bce-3007-4d67-814f-9533805e9d84 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:18.789526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 20/40] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:18.791211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04beb4c-607d-4cfb-aab9-62a0b70be3ff None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:18.791702 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04beb4c-607d-4cfb-aab9-62a0b70be3ff None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:18.792119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04beb4c-607d-4cfb-aab9-62a0b70be3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:18.813493 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-c04beb4c-607d-4cfb-aab9-62a0b70be3ff None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:18.814733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c04beb4c-607d-4cfb-aab9-62a0b70be3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:18.878661 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:18.890541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 21/41] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:18.972005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc11bce-3007-4d67-814f-9533805e9d84 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:18.996797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:19.009538 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160ae44-9f04-4043-ab5a-65cfcc00991b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:19.010184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160ae44-9f04-4043-ab5a-65cfcc00991b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:19.010467 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160ae44-9f04-4043-ab5a-65cfcc00991b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:19.010778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160ae44-9f04-4043-ab5a-65cfcc00991b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:19.012012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160ae44-9f04-4043-ab5a-65cfcc00991b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:19.013489 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160ae44-9f04-4043-ab5a-65cfcc00991b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:19.015289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 21/42] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:19.021181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04beb4c-607d-4cfb-aab9-62a0b70be3ff None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:19.050009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160ae44-9f04-4043-ab5a-65cfcc00991b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:19.142054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04beb4c-607d-4cfb-aab9-62a0b70be3ff None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:19.164090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160ae44-9f04-4043-ab5a-65cfcc00991b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:19.175231 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b38ae3-26fe-45af-a0ce-5ea036215d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:19.175905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b38ae3-26fe-45af-a0ce-5ea036215d2a None admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:19.176186 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b38ae3-26fe-45af-a0ce-5ea036215d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:19.176463 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b38ae3-26fe-45af-a0ce-5ea036215d2a None admin] PATH_INFO: `/v3/domains/default/users/94cbbd1503324af1af6df581aea570b9/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:19.178636 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b38ae3-26fe-45af-a0ce-5ea036215d2a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=94cbbd1503324af1af6df581aea570b9, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:19.180340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b38ae3-26fe-45af-a0ce-5ea036215d2a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:19.197869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 22/43] 10.222.0.23 () {58 vars in 1361 bytes} [Fri Sep 9 16:17:19 2022] PUT /identity/v3/domains/default/users/94cbbd1503324af1af6df581aea570b9/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:17:19.198749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b38ae3-26fe-45af-a0ce-5ea036215d2a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:19.205556 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10148693-d6c5-4e3a-922a-2d038b6f4b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:19.206217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10148693-d6c5-4e3a-922a-2d038b6f4b48 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:19.206489 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10148693-d6c5-4e3a-922a-2d038b6f4b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:19.206792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10148693-d6c5-4e3a-922a-2d038b6f4b48 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:19.207483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10148693-d6c5-4e3a-922a-2d038b6f4b48 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:19.208929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10148693-d6c5-4e3a-922a-2d038b6f4b48 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:19.225625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 22/44] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Sep 9 16:17:19 2022] POST /identity/v3/roles => generated 223 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:17:19.276983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:19.336498 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b38ae3-26fe-45af-a0ce-5ea036215d2a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:19.380096 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10943e21-dbc3-4834-833b-276afb930c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:19.381148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10943e21-dbc3-4834-833b-276afb930c07 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:19.381148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10943e21-dbc3-4834-833b-276afb930c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:19.381567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10943e21-dbc3-4834-833b-276afb930c07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:19.383091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10943e21-dbc3-4834-833b-276afb930c07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:19.384653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10943e21-dbc3-4834-833b-276afb930c07 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:19.386689 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 23/45] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:19.392544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10148693-d6c5-4e3a-922a-2d038b6f4b48 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:19.451155 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:19.507485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10148693-d6c5-4e3a-922a-2d038b6f4b48 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:19.541982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63714d12-00be-4bd1-966f-52a40b475cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:19.542626 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63714d12-00be-4bd1-966f-52a40b475cb9 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:19.542907 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63714d12-00be-4bd1-966f-52a40b475cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:19.543177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63714d12-00be-4bd1-966f-52a40b475cb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:19.543830 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63714d12-00be-4bd1-966f-52a40b475cb9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:19.545291 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63714d12-00be-4bd1-966f-52a40b475cb9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:19.575819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 23/46] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Sep 9 16:17:19 2022] POST /identity/v3/projects => generated 312 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:17:21.388351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd2e5d-8503-46a1-8633-49ef06fc21c9 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:21.388892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd2e5d-8503-46a1-8633-49ef06fc21c9 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:21.388892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd2e5d-8503-46a1-8633-49ef06fc21c9 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:21.390241 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 24/47] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:21.395554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630249b-36b9-477e-94e4-05ac66e22b78 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:21.395969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630249b-36b9-477e-94e4-05ac66e22b78 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:21.396236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630249b-36b9-477e-94e4-05ac66e22b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:21.415305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5630249b-36b9-477e-94e4-05ac66e22b78 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:21.416848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5630249b-36b9-477e-94e4-05ac66e22b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:21.492054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 24/48] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:21.731220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861e7b8-9b0b-4c2b-8fa5-2ed2600dc888 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:21.731424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861e7b8-9b0b-4c2b-8fa5-2ed2600dc888 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:21.731701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861e7b8-9b0b-4c2b-8fa5-2ed2600dc888 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:21.733114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 25/49] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:21.738687 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46122ef-80cf-4e62-833f-fa57be8649de None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:21.738986 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46122ef-80cf-4e62-833f-fa57be8649de None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:21.739226 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46122ef-80cf-4e62-833f-fa57be8649de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:21.758082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861e7b8-9b0b-4c2b-8fa5-2ed2600dc888 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:21.758684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b46122ef-80cf-4e62-833f-fa57be8649de None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:21.759901 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b46122ef-80cf-4e62-833f-fa57be8649de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:21.787343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:21.828855 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 25/50] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:21.862042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861e7b8-9b0b-4c2b-8fa5-2ed2600dc888 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:21.881671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:21.896342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce05f7e-5288-480e-8a9d-0f8f7ca8649b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:21.896977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce05f7e-5288-480e-8a9d-0f8f7ca8649b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:21.897263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce05f7e-5288-480e-8a9d-0f8f7ca8649b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:21.897537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce05f7e-5288-480e-8a9d-0f8f7ca8649b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:21.908089 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce05f7e-5288-480e-8a9d-0f8f7ca8649b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-bce05f7e-5288-480e-8a9d-0f8f7ca8649b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:21.912977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:21.913971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:21.914742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:21.914742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:21.914742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:21.914742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:21.914742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:21.914742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:21.914742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:21.914742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:21.914742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:21.914742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:21.914742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:21.914742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:21.914742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 26/51] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:17:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:21.921478 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46122ef-80cf-4e62-833f-fa57be8649de None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:22.025119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46122ef-80cf-4e62-833f-fa57be8649de None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:22.060301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a00023-1c7e-4ad0-b70a-a769f7c5b874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:22.060916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a00023-1c7e-4ad0-b70a-a769f7c5b874 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:22.061200 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a00023-1c7e-4ad0-b70a-a769f7c5b874 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:22.061478 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a00023-1c7e-4ad0-b70a-a769f7c5b874 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:22.062127 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a00023-1c7e-4ad0-b70a-a769f7c5b874 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:22.062434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a00023-1c7e-4ad0-b70a-a769f7c5b874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:22.063855 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a00023-1c7e-4ad0-b70a-a769f7c5b874 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:22.075548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 26/52] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:22.078743 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce05f7e-5288-480e-8a9d-0f8f7ca8649b None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:22.091442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a00023-1c7e-4ad0-b70a-a769f7c5b874 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:22.118378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:22.183177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce05f7e-5288-480e-8a9d-0f8f7ca8649b None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:22.191602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a00023-1c7e-4ad0-b70a-a769f7c5b874 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:22.216119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df39afe-d485-4476-ba28-29f9ba26b499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:22.216756 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df39afe-d485-4476-ba28-29f9ba26b499 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:22.217035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df39afe-d485-4476-ba28-29f9ba26b499 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:22.217310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df39afe-d485-4476-ba28-29f9ba26b499 None admin] PATH_INFO: `/v3/users/94cbbd1503324af1af6df581aea570b9` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:22.218447 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df39afe-d485-4476-ba28-29f9ba26b499 None admin] RBAC: Authorizing `identity:get_user(user_id=94cbbd1503324af1af6df581aea570b9)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:22.219955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df39afe-d485-4476-ba28-29f9ba26b499 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:22.221531 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 27/53] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:22 2022] GET /identity/v3/users/94cbbd1503324af1af6df581aea570b9 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:22.225887 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0a560-54f0-4034-a6a5-d06362962e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:22.226600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0a560-54f0-4034-a6a5-d06362962e23 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:22.226889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0a560-54f0-4034-a6a5-d06362962e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:22.227016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df39afe-d485-4476-ba28-29f9ba26b499 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:22.227163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0a560-54f0-4034-a6a5-d06362962e23 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:22.228296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0a560-54f0-4034-a6a5-d06362962e23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:22.229774 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0a560-54f0-4034-a6a5-d06362962e23 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:22.231381 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 27/54] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:22.236837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0a560-54f0-4034-a6a5-d06362962e23 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:22.337809 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:22.347557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0a560-54f0-4034-a6a5-d06362962e23 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:22.382548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b32a4-4bf8-4ef6-8ce1-93caec8a9c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:22.383236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b32a4-4bf8-4ef6-8ce1-93caec8a9c98 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:22.383530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b32a4-4bf8-4ef6-8ce1-93caec8a9c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:22.383812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b32a4-4bf8-4ef6-8ce1-93caec8a9c98 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:22.383904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:22.384458 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b32a4-4bf8-4ef6-8ce1-93caec8a9c98 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:22.386066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b32a4-4bf8-4ef6-8ce1-93caec8a9c98 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:22.414278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 28/55] 10.222.0.23 () {60 vars in 1140 bytes} [Fri Sep 9 16:17:22 2022] POST /identity/v3/projects => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:17:22.436819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df39afe-d485-4476-ba28-29f9ba26b499 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:22.475436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e026a6-43e3-4e65-8eea-53b079e7ad7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:22.476150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e026a6-43e3-4e65-8eea-53b079e7ad7e None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:22.476432 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e026a6-43e3-4e65-8eea-53b079e7ad7e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:22.476705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e026a6-43e3-4e65-8eea-53b079e7ad7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:22.477952 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e026a6-43e3-4e65-8eea-53b079e7ad7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:22.479487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e026a6-43e3-4e65-8eea-53b079e7ad7e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:22.481116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 28/56] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:22.487582 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b32a4-4bf8-4ef6-8ce1-93caec8a9c98 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:22.593803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b32a4-4bf8-4ef6-8ce1-93caec8a9c98 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:22.628786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62701106-a77e-49c0-b859-493645bef246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:22.629407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62701106-a77e-49c0-b859-493645bef246 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:22.629688 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62701106-a77e-49c0-b859-493645bef246 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:22.629957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62701106-a77e-49c0-b859-493645bef246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:22.630713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62701106-a77e-49c0-b859-493645bef246 None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.domain.id=default, user.id=94cbbd1503324af1af6df581aea570b9) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:22.631019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62701106-a77e-49c0-b859-493645bef246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:22.632522 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62701106-a77e-49c0-b859-493645bef246 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:22.638556 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 29/57] 10.222.0.23 () {56 vars in 1322 bytes} [Fri Sep 9 16:17:22 2022] GET /identity/v3/role_assignments?user.id=94cbbd1503324af1af6df581aea570b9&scope.domain.id=default&role.id=c6da7ba2be504943af29338ee8855f05 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:24.554244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a59c6-c92e-42f6-88c3-0cc3450ef80e None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:24.555180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a59c6-c92e-42f6-88c3-0cc3450ef80e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:24.555180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a59c6-c92e-42f6-88c3-0cc3450ef80e None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:24.557418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 29/58] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:24.563701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550421b-e7c4-4b03-8328-68516882d6bf None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:24.564118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550421b-e7c4-4b03-8328-68516882d6bf None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:24.564306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550421b-e7c4-4b03-8328-68516882d6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:24.588119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b550421b-e7c4-4b03-8328-68516882d6bf None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:24.590009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b550421b-e7c4-4b03-8328-68516882d6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:24.593197 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:24.671651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 30/59] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:24.933778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a59c6-c92e-42f6-88c3-0cc3450ef80e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:25.042819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a59c6-c92e-42f6-88c3-0cc3450ef80e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:25.061193 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:25.080712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0cade-2913-48ec-aef1-96d01d65b11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:25.081337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0cade-2913-48ec-aef1-96d01d65b11c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:25.081613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0cade-2913-48ec-aef1-96d01d65b11c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:25.081881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0cade-2913-48ec-aef1-96d01d65b11c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:25.083069 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0cade-2913-48ec-aef1-96d01d65b11c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:25.084521 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0cade-2913-48ec-aef1-96d01d65b11c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:25.086117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 30/60] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:25.091878 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550421b-e7c4-4b03-8328-68516882d6bf None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:25.208349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550421b-e7c4-4b03-8328-68516882d6bf None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:25.248699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd54b5c5-bc7c-4e85-acc3-3960e90ae369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:25.249314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54b5c5-bc7c-4e85-acc3-3960e90ae369 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:25.249593 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54b5c5-bc7c-4e85-acc3-3960e90ae369 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:25.249864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54b5c5-bc7c-4e85-acc3-3960e90ae369 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:25.250543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd54b5c5-bc7c-4e85-acc3-3960e90ae369 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:25.252013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd54b5c5-bc7c-4e85-acc3-3960e90ae369 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:25.266617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:25.281441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 31/61] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:17:25 2022] POST /identity/v3/users => generated 275 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:17:28.473579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41e26f-0822-4bfd-b841-697a5b8e9413 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:28.474474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41e26f-0822-4bfd-b841-697a5b8e9413 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:28.474909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41e26f-0822-4bfd-b841-697a5b8e9413 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:28.476693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 31/62] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:28.482743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4975c-f3d7-435c-900f-652db99b435a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:28.483589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4975c-f3d7-435c-900f-652db99b435a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:28.484042 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4975c-f3d7-435c-900f-652db99b435a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:28.508612 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b4e4975c-f3d7-435c-900f-652db99b435a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:28.514539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b4e4975c-f3d7-435c-900f-652db99b435a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:28.615231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 32/63] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:28.620548 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb20c45-de55-4bc3-80b9-d18950e12de5 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:28.621025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb20c45-de55-4bc3-80b9-d18950e12de5 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:28.621440 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb20c45-de55-4bc3-80b9-d18950e12de5 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:28.623202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 32/64] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:28.628201 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd6862-9ab2-409e-974d-511afa46b574 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:28.628643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd6862-9ab2-409e-974d-511afa46b574 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:28.629075 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd6862-9ab2-409e-974d-511afa46b574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:28.655931 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-47bd6862-9ab2-409e-974d-511afa46b574 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:28.657186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-47bd6862-9ab2-409e-974d-511afa46b574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:28.739161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 33/65] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:17:28.899364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b74b3-f1ff-4589-9d21-de053eeccf93 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:28.900062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b74b3-f1ff-4589-9d21-de053eeccf93 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:28.900575 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b74b3-f1ff-4589-9d21-de053eeccf93 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:28.902148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 33/66] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:28.907850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e3540-16af-422c-a5c5-45038b60c6dd None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:28.908369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e3540-16af-422c-a5c5-45038b60c6dd None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:28.908839 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e3540-16af-422c-a5c5-45038b60c6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:28.939850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-cb6e3540-16af-422c-a5c5-45038b60c6dd None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:28.941948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b74b3-f1ff-4589-9d21-de053eeccf93 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:28.947287 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cb6e3540-16af-422c-a5c5-45038b60c6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:29.044497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:29.069203 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 34/67] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:29.074707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e3540-16af-422c-a5c5-45038b60c6dd None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:29.126150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b74b3-f1ff-4589-9d21-de053eeccf93 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:29.164592 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90dab66-3a82-4a83-aa18-7adb5b20eb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:29.165328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dab66-3a82-4a83-aa18-7adb5b20eb1e None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:29.165609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dab66-3a82-4a83-aa18-7adb5b20eb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:29.165937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dab66-3a82-4a83-aa18-7adb5b20eb1e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:29.177982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dab66-3a82-4a83-aa18-7adb5b20eb1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-c90dab66-3a82-4a83-aa18-7adb5b20eb1e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:29.184433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:29.185466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:29.186386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:29.186386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:29.186386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:29.186386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:29.186386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:29.186386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:29.186386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:29.186386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:29.186386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:29.186386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:29.186386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:29.186386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:29.189427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 34/68] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:17:29.190845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62f412-d3ba-4e7b-bd8d-e934b19a9ccd None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:29.191239 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e3540-16af-422c-a5c5-45038b60c6dd None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:29.191414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62f412-d3ba-4e7b-bd8d-e934b19a9ccd None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:29.191810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62f412-d3ba-4e7b-bd8d-e934b19a9ccd None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:29.193657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 35/69] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:29.197895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62f412-d3ba-4e7b-bd8d-e934b19a9ccd None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:29.239652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a590b2-33f5-4d55-9664-7abcf182cce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:29.240317 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a590b2-33f5-4d55-9664-7abcf182cce2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:29.240599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a590b2-33f5-4d55-9664-7abcf182cce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:29.240879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a590b2-33f5-4d55-9664-7abcf182cce2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:29.251801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:29.254648 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a590b2-33f5-4d55-9664-7abcf182cce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-27a590b2-33f5-4d55-9664-7abcf182cce2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:29.261168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:29.262081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:29.262879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:29.262879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:29.262879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:29.262879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:29.262879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:29.262879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:29.262879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:29.262879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:29.262879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:29.262879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:29.262879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:29.262879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:29.262879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 35/70] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:17:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:29.268709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422937c-7200-4722-8539-042ccf0c41da None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:29.268709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422937c-7200-4722-8539-042ccf0c41da None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:29.268709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422937c-7200-4722-8539-042ccf0c41da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:29.289329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1422937c-7200-4722-8539-042ccf0c41da None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:29.290889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1422937c-7200-4722-8539-042ccf0c41da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:29.329167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62f412-d3ba-4e7b-bd8d-e934b19a9ccd None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:29.342871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:29.370061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeea38-6e3a-4a54-8357-59fbb57018da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:29.370717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeea38-6e3a-4a54-8357-59fbb57018da None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:29.371004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeea38-6e3a-4a54-8357-59fbb57018da None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:29.371345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeea38-6e3a-4a54-8357-59fbb57018da None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:29.371952 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eeea38-6e3a-4a54-8357-59fbb57018da None admin] RBAC: Adding query filter params (name=member) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:29.372260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eeea38-6e3a-4a54-8357-59fbb57018da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:29.379036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 36/71] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:29.379180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eeea38-6e3a-4a54-8357-59fbb57018da None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:29.379272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422937c-7200-4722-8539-042ccf0c41da None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:29.386929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 36/72] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:29.387695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeea38-6e3a-4a54-8357-59fbb57018da None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:29.480548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422937c-7200-4722-8539-042ccf0c41da None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:29.495308 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeea38-6e3a-4a54-8357-59fbb57018da None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:29.505065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:29.516773 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166aab2e-25fe-4e22-8ea9-8bd73049e895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:29.517399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166aab2e-25fe-4e22-8ea9-8bd73049e895 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:29.517682 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166aab2e-25fe-4e22-8ea9-8bd73049e895 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:29.517954 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166aab2e-25fe-4e22-8ea9-8bd73049e895 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:29.518663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166aab2e-25fe-4e22-8ea9-8bd73049e895 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:29.518968 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166aab2e-25fe-4e22-8ea9-8bd73049e895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:29.520354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166aab2e-25fe-4e22-8ea9-8bd73049e895 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:29.528682 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 37/73] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:29.529741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a85685-2b8c-419b-bae4-10103fa9251d None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:29.530032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a85685-2b8c-419b-bae4-10103fa9251d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:29.530308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a85685-2b8c-419b-bae4-10103fa9251d None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:29.531537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 38/74] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:29.535281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a85685-2b8c-419b-bae4-10103fa9251d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:29.541160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d017ae5-6233-4d73-b01c-2022261c06c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:29.541810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d017ae5-6233-4d73-b01c-2022261c06c6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:29.542084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d017ae5-6233-4d73-b01c-2022261c06c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:29.542356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d017ae5-6233-4d73-b01c-2022261c06c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:29.552656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d017ae5-6233-4d73-b01c-2022261c06c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-2d017ae5-6233-4d73-b01c-2022261c06c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:29.557486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:29.558318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:29.559088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:29.559088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:29.559088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:29.559088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:29.559088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:29.559088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:29.559088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:29.559088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:29.559088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:29.559088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:29.559088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:17:29.559088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:29.559088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 37/75] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:17:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:29.559703 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d017ae5-6233-4d73-b01c-2022261c06c6 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:29.576238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:29.642613 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a85685-2b8c-419b-bae4-10103fa9251d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:29.667113 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d017ae5-6233-4d73-b01c-2022261c06c6 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:29.675708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55675874-30b4-4ad9-9bf2-c9fa54a29a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:29.676347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55675874-30b4-4ad9-9bf2-c9fa54a29a1b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:29.676630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55675874-30b4-4ad9-9bf2-c9fa54a29a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:29.676900 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55675874-30b4-4ad9-9bf2-c9fa54a29a1b None admin] PATH_INFO: `/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:29.689975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55675874-30b4-4ad9-9bf2-c9fa54a29a1b None admin] RBAC: Authorizing `identity:get_user(user_id=6ca6948ab5ab429ab41c690eaa6cc6c5)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:29.691500 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55675874-30b4-4ad9-9bf2-c9fa54a29a1b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:29.693506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 39/76] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:29 2022] GET /identity/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:29.694670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ddec5-c9b5-4971-b350-9981c19c38c5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:29.694993 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ddec5-c9b5-4971-b350-9981c19c38c5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:29.695273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ddec5-c9b5-4971-b350-9981c19c38c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:29.700015 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530452f-df3e-43ad-89a4-c1b697410161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:29.700658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530452f-df3e-43ad-89a4-c1b697410161 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:29.700942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530452f-df3e-43ad-89a4-c1b697410161 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:29.701220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530452f-df3e-43ad-89a4-c1b697410161 None admin] PATH_INFO: `/v3/users/94cbbd1503324af1af6df581aea570b9` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:29.702352 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530452f-df3e-43ad-89a4-c1b697410161 None admin] RBAC: Authorizing `identity:get_user(user_id=94cbbd1503324af1af6df581aea570b9)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:29.703861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530452f-df3e-43ad-89a4-c1b697410161 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:29.705534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 38/77] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:29 2022] GET /identity/v3/users/94cbbd1503324af1af6df581aea570b9 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:29.706305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530452f-df3e-43ad-89a4-c1b697410161 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:29.715622 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-316ddec5-c9b5-4971-b350-9981c19c38c5 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:29.716816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-316ddec5-c9b5-4971-b350-9981c19c38c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:29.739454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:29.779938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 40/78] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:29.780865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ddec5-c9b5-4971-b350-9981c19c38c5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:29.807131 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:29.818585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530452f-df3e-43ad-89a4-c1b697410161 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:29.853572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285d5a7-ec14-428b-923e-13a6945a1d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:29.854202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285d5a7-ec14-428b-923e-13a6945a1d03 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:29.854487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285d5a7-ec14-428b-923e-13a6945a1d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:29.854796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285d5a7-ec14-428b-923e-13a6945a1d03 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:29.855474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285d5a7-ec14-428b-923e-13a6945a1d03 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:29.855782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285d5a7-ec14-428b-923e-13a6945a1d03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:29.857169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285d5a7-ec14-428b-923e-13a6945a1d03 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:29.865279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 39/79] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:17:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:29.866054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285d5a7-ec14-428b-923e-13a6945a1d03 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:29.894576 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ddec5-c9b5-4971-b350-9981c19c38c5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:29.930623 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571eb157-9a05-40b4-81ba-72ef66af3710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:29.931269 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eb157-9a05-40b4-81ba-72ef66af3710 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:29.931555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eb157-9a05-40b4-81ba-72ef66af3710 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:29.931829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eb157-9a05-40b4-81ba-72ef66af3710 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:29.941768 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571eb157-9a05-40b4-81ba-72ef66af3710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-571eb157-9a05-40b4-81ba-72ef66af3710 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:29.947028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:29.948062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:29.948842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:29.948842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:29.948842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:29.948842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:29.948842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:29.948842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:29.948842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:29.948842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:29.948842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:29.948842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:29.948842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:29.948842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:29.948842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 41/80] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:29.949449 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571eb157-9a05-40b4-81ba-72ef66af3710 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:29.973133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:29.982037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285d5a7-ec14-428b-923e-13a6945a1d03 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:30.018867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3181898-fb80-4bd4-b2f1-d976de49f903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:30.019488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3181898-fb80-4bd4-b2f1-d976de49f903 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:30.019773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3181898-fb80-4bd4-b2f1-d976de49f903 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:30.020050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3181898-fb80-4bd4-b2f1-d976de49f903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:30.039281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:30.055867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571eb157-9a05-40b4-81ba-72ef66af3710 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:30.088763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3181898-fb80-4bd4-b2f1-d976de49f903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:30.090244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3181898-fb80-4bd4-b2f1-d976de49f903 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:30.091412 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e183a-8b71-47a5-aeff-60948fd7d163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:30.092036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e183a-8b71-47a5-aeff-60948fd7d163 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:30.092312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e183a-8b71-47a5-aeff-60948fd7d163 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:30.092582 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e183a-8b71-47a5-aeff-60948fd7d163 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:30.157849 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e183a-8b71-47a5-aeff-60948fd7d163 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:30.159293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e183a-8b71-47a5-aeff-60948fd7d163 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:30.179522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:30.186618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 40/81] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:17:30.187441 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3181898-fb80-4bd4-b2f1-d976de49f903 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:30.246206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:30.263313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 42/82] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:17:30.264155 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e183a-8b71-47a5-aeff-60948fd7d163 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:30.284164 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3181898-fb80-4bd4-b2f1-d976de49f903 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:30.316606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e043c58-6849-450f-8f32-79c60a74a927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:30.317593 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e043c58-6849-450f-8f32-79c60a74a927 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:30.317869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e043c58-6849-450f-8f32-79c60a74a927 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:30.318138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e043c58-6849-450f-8f32-79c60a74a927 None admin] PATH_INFO: `/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:30.319348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e043c58-6849-450f-8f32-79c60a74a927 None admin] RBAC: Authorizing `identity:get_user(user_id=6ca6948ab5ab429ab41c690eaa6cc6c5)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:30.320842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e043c58-6849-450f-8f32-79c60a74a927 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:30.322421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 41/83] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:30 2022] GET /identity/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5 => generated 275 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:30.326978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e043c58-6849-450f-8f32-79c60a74a927 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:30.363573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e183a-8b71-47a5-aeff-60948fd7d163 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:30.383033 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:30.396922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1bb66-ac80-403d-b1b6-d430464ed1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:30.397538 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1bb66-ac80-403d-b1b6-d430464ed1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:30.397824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1bb66-ac80-403d-b1b6-d430464ed1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:30.398130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1bb66-ac80-403d-b1b6-d430464ed1d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:30.398831 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1bb66-ac80-403d-b1b6-d430464ed1d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:30.399139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1bb66-ac80-403d-b1b6-d430464ed1d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:30.400552 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1bb66-ac80-403d-b1b6-d430464ed1d9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:30.408947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 43/84] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:30.409689 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1bb66-ac80-403d-b1b6-d430464ed1d9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:30.420432 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e043c58-6849-450f-8f32-79c60a74a927 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:30.453982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35458d-9a16-4615-85bf-d2f697253cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:30.454768 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35458d-9a16-4615-85bf-d2f697253cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:30.455073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35458d-9a16-4615-85bf-d2f697253cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:30.455345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35458d-9a16-4615-85bf-d2f697253cb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:30.456448 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35458d-9a16-4615-85bf-d2f697253cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:30.456809 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:30.457896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35458d-9a16-4615-85bf-d2f697253cb4 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:30.459533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 42/85] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:30.460282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35458d-9a16-4615-85bf-d2f697253cb4 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:30.512109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1bb66-ac80-403d-b1b6-d430464ed1d9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:30.545895 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d9e08-00b5-4e4b-aff8-b009a8c26f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:30.546542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d9e08-00b5-4e4b-aff8-b009a8c26f2f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:30.546833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d9e08-00b5-4e4b-aff8-b009a8c26f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:30.547093 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d9e08-00b5-4e4b-aff8-b009a8c26f2f None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:30.548229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d9e08-00b5-4e4b-aff8-b009a8c26f2f None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:30.549706 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d9e08-00b5-4e4b-aff8-b009a8c26f2f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:30.551388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 44/86] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:30 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:30.552153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d9e08-00b5-4e4b-aff8-b009a8c26f2f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:30.560172 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35458d-9a16-4615-85bf-d2f697253cb4 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:30.590073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:30.595761 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd1048-4f8b-4a72-a7b6-07e8db5de8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:30.596428 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd1048-4f8b-4a72-a7b6-07e8db5de8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:30.596692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd1048-4f8b-4a72-a7b6-07e8db5de8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:30.596975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd1048-4f8b-4a72-a7b6-07e8db5de8e2 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:30.598117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd1048-4f8b-4a72-a7b6-07e8db5de8e2 None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:30.599612 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd1048-4f8b-4a72-a7b6-07e8db5de8e2 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:30.601292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 43/87] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:30 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:30.602044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd1048-4f8b-4a72-a7b6-07e8db5de8e2 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:30.654433 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d9e08-00b5-4e4b-aff8-b009a8c26f2f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:30.667837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:30.688527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fda149-3181-4dc4-8dbb-73229d82984d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:30.689131 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda149-3181-4dc4-8dbb-73229d82984d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:30.689416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda149-3181-4dc4-8dbb-73229d82984d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:30.689683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda149-3181-4dc4-8dbb-73229d82984d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:30.699100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd1048-4f8b-4a72-a7b6-07e8db5de8e2 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:30.731750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d550f93-c36d-4940-b151-8b94bcf1afa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:30.732378 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d550f93-c36d-4940-b151-8b94bcf1afa7 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:30.732662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d550f93-c36d-4940-b151-8b94bcf1afa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:30.732940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d550f93-c36d-4940-b151-8b94bcf1afa7 None admin] PATH_INFO: `/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:30.734073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d550f93-c36d-4940-b151-8b94bcf1afa7 None admin] RBAC: Authorizing `identity:get_user(user_id=6ca6948ab5ab429ab41c690eaa6cc6c5)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:30.735570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d550f93-c36d-4940-b151-8b94bcf1afa7 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:30.737161 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 44/88] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:30 2022] GET /identity/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5 => generated 275 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:30.742050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d550f93-c36d-4940-b151-8b94bcf1afa7 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:30.758002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fda149-3181-4dc4-8dbb-73229d82984d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:30.759553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fda149-3181-4dc4-8dbb-73229d82984d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:30.801258 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:30.836473 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d550f93-c36d-4940-b151-8b94bcf1afa7 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:30.855908 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 45/89] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:17:30.856758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fda149-3181-4dc4-8dbb-73229d82984d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:30.869044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac3c99-27fd-48ec-a6aa-eb4ab84b4f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:30.869657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac3c99-27fd-48ec-a6aa-eb4ab84b4f6f None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:30.869920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac3c99-27fd-48ec-a6aa-eb4ab84b4f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:30.870189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac3c99-27fd-48ec-a6aa-eb4ab84b4f6f None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:30.870859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac3c99-27fd-48ec-a6aa-eb4ab84b4f6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:30.871198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:30.872294 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac3c99-27fd-48ec-a6aa-eb4ab84b4f6f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:30.899313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 45/90] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:17:30 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:17:30.900179 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac3c99-27fd-48ec-a6aa-eb4ab84b4f6f None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:30.973677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fda149-3181-4dc4-8dbb-73229d82984d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:31.016886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575204c5-595f-4ef8-8d6d-3146edc66e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:31.017517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575204c5-595f-4ef8-8d6d-3146edc66e76 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:31.017799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575204c5-595f-4ef8-8d6d-3146edc66e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:31.018069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575204c5-595f-4ef8-8d6d-3146edc66e76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:31.018350 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac3c99-27fd-48ec-a6aa-eb4ab84b4f6f None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:31.018841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575204c5-595f-4ef8-8d6d-3146edc66e76 None admin] RBAC: Adding query filter params (role.id=9e02fa627e5c4388b0aa7182f1707895, scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b, user.id=6ca6948ab5ab429ab41c690eaa6cc6c5) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:31.019148 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575204c5-595f-4ef8-8d6d-3146edc66e76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:31.021104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575204c5-595f-4ef8-8d6d-3146edc66e76 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:31.027450 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 46/91] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:30 2022] GET /identity/v3/role_assignments?user.id=6ca6948ab5ab429ab41c690eaa6cc6c5&scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b&role.id=9e02fa627e5c4388b0aa7182f1707895 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:31.028141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575204c5-595f-4ef8-8d6d-3146edc66e76 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:31.062100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:31.079047 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2572f-03c1-4e1e-878d-2a54acf7ec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:31.079852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2572f-03c1-4e1e-878d-2a54acf7ec96 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:31.080573 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2572f-03c1-4e1e-878d-2a54acf7ec96 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:31.081177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2572f-03c1-4e1e-878d-2a54acf7ec96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:31.082322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2572f-03c1-4e1e-878d-2a54acf7ec96 None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b, user.id=94cbbd1503324af1af6df581aea570b9) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:31.083431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2572f-03c1-4e1e-878d-2a54acf7ec96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:31.085674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2572f-03c1-4e1e-878d-2a54acf7ec96 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:31.102532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 46/92] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:30 2022] GET /identity/v3/role_assignments?user.id=94cbbd1503324af1af6df581aea570b9&scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b&role.id=c6da7ba2be504943af29338ee8855f05 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:31.102532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2572f-03c1-4e1e-878d-2a54acf7ec96 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:31.146935 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:31.177780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575204c5-595f-4ef8-8d6d-3146edc66e76 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:31.229613 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47371fc-3239-4be4-88da-c88f50c9adb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:31.230216 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47371fc-3239-4be4-88da-c88f50c9adb9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:31.230545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47371fc-3239-4be4-88da-c88f50c9adb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:31.230811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47371fc-3239-4be4-88da-c88f50c9adb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:31.236927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2572f-03c1-4e1e-878d-2a54acf7ec96 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:31.284117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaec41-6417-449c-988a-a68a5afa86bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:31.284795 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaec41-6417-449c-988a-a68a5afa86bc None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:31.285752 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaec41-6417-449c-988a-a68a5afa86bc None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:31.286032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaec41-6417-449c-988a-a68a5afa86bc None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:31.287278 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeaec41-6417-449c-988a-a68a5afa86bc None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:31.289704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeaec41-6417-449c-988a-a68a5afa86bc None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:31.291821 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 47/93] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:31 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:31.298181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaec41-6417-449c-988a-a68a5afa86bc None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:31.329483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47371fc-3239-4be4-88da-c88f50c9adb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:31.330936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47371fc-3239-4be4-88da-c88f50c9adb9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:31.378097 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:31.420656 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:31.431282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaec41-6417-449c-988a-a68a5afa86bc None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:31.448122 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 47/94] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:17:31.455655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47371fc-3239-4be4-88da-c88f50c9adb9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:31.476443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a781d-2b8c-4830-889b-adc2eae3e656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:31.477245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a781d-2b8c-4830-889b-adc2eae3e656 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:31.477528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a781d-2b8c-4830-889b-adc2eae3e656 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:31.477795 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a781d-2b8c-4830-889b-adc2eae3e656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:31.478971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a781d-2b8c-4830-889b-adc2eae3e656 None admin] RBAC: Adding query filter params (role.id=0d40b3176f78460f9c4a29c1e9ef843e, scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b, user.id=6ca6948ab5ab429ab41c690eaa6cc6c5) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:31.479561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a781d-2b8c-4830-889b-adc2eae3e656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:31.481146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a781d-2b8c-4830-889b-adc2eae3e656 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:31.489279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 48/95] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:31 2022] GET /identity/v3/role_assignments?user.id=6ca6948ab5ab429ab41c690eaa6cc6c5&scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b&role.id=0d40b3176f78460f9c4a29c1e9ef843e => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:31.594853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47371fc-3239-4be4-88da-c88f50c9adb9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:31.640261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16616cec-7f2e-4785-8c7e-fd5d812728b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:31.641310 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16616cec-7f2e-4785-8c7e-fd5d812728b4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:31.641801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16616cec-7f2e-4785-8c7e-fd5d812728b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:31.642266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16616cec-7f2e-4785-8c7e-fd5d812728b4 None admin] PATH_INFO: `/v3/projects/a2abfc5ca6da4956ac25b8040e6b4e58` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:31.643837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16616cec-7f2e-4785-8c7e-fd5d812728b4 None admin] RBAC: Authorizing `identity:get_project(project_id=a2abfc5ca6da4956ac25b8040e6b4e58)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:31.645621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16616cec-7f2e-4785-8c7e-fd5d812728b4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:31.647698 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 48/96] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:31 2022] GET /identity/v3/projects/a2abfc5ca6da4956ac25b8040e6b4e58 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:31.654112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a781d-2b8c-4830-889b-adc2eae3e656 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:31.798933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a781d-2b8c-4830-889b-adc2eae3e656 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:31.827579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:31.848713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad1f4b-ae5e-4b44-9290-52cb3ebfbf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:31.849676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad1f4b-ae5e-4b44-9290-52cb3ebfbf92 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:31.850149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad1f4b-ae5e-4b44-9290-52cb3ebfbf92 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:31.850640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad1f4b-ae5e-4b44-9290-52cb3ebfbf92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:31.851663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad1f4b-ae5e-4b44-9290-52cb3ebfbf92 None admin] RBAC: Adding query filter params (role.id=9e02fa627e5c4388b0aa7182f1707895, scope.project.id=a2abfc5ca6da4956ac25b8040e6b4e58, user.id=6ca6948ab5ab429ab41c690eaa6cc6c5) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:31.852267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad1f4b-ae5e-4b44-9290-52cb3ebfbf92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:31.853920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad1f4b-ae5e-4b44-9290-52cb3ebfbf92 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:31.862751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 49/97] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:31 2022] GET /identity/v3/role_assignments?user.id=6ca6948ab5ab429ab41c690eaa6cc6c5&scope.project.id=a2abfc5ca6da4956ac25b8040e6b4e58&role.id=9e02fa627e5c4388b0aa7182f1707895 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:33.774650 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6e617-1ace-4088-a013-662fd933ed7a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:33.775672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6e617-1ace-4088-a013-662fd933ed7a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:33.776163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6e617-1ace-4088-a013-662fd933ed7a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:33.777741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 49/98] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:33.783254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241deea-6543-41fc-976a-22d1507b4b99 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:33.784070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241deea-6543-41fc-976a-22d1507b4b99 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:33.784561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241deea-6543-41fc-976a-22d1507b4b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:33.809982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-9241deea-6543-41fc-976a-22d1507b4b99 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:33.811530 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-9241deea-6543-41fc-976a-22d1507b4b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:33.898266 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 50/99] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:34.002721 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353e2fb-4e50-4e23-8748-bc4abcff7ac9 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:34.003370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353e2fb-4e50-4e23-8748-bc4abcff7ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:34.003836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353e2fb-4e50-4e23-8748-bc4abcff7ac9 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:34.005397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 50/100] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:34.010770 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac5da5-f331-4434-82ca-c1147933e330 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:34.011290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac5da5-f331-4434-82ca-c1147933e330 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:34.011774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac5da5-f331-4434-82ca-c1147933e330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:34.036357 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-cbac5da5-f331-4434-82ca-c1147933e330 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:34.037837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cbac5da5-f331-4434-82ca-c1147933e330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:34.125017 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 51/101] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:34.304756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353e2fb-4e50-4e23-8748-bc4abcff7ac9 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:34.311352 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e8c6c-a5c3-499e-a157-5ed2db0ea491 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:34.311644 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e8c6c-a5c3-499e-a157-5ed2db0ea491 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:34.311925 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e8c6c-a5c3-499e-a157-5ed2db0ea491 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:34.313436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 52/102] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:17:34.328394 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11691f-f381-4656-a192-fc9c416891a0 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:34.328974 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11691f-f381-4656-a192-fc9c416891a0 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:34.329260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11691f-f381-4656-a192-fc9c416891a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:34.358899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-7b11691f-f381-4656-a192-fc9c416891a0 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:34.360278 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7b11691f-f381-4656-a192-fc9c416891a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:34.361102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:34.435650 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:34.446287 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353e2fb-4e50-4e23-8748-bc4abcff7ac9 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:34.499434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9f9a2-2ca0-4213-b3cc-3b6ffd105539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:34.500002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9f9a2-2ca0-4213-b3cc-3b6ffd105539 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:34.500281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9f9a2-2ca0-4213-b3cc-3b6ffd105539 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:34.500554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9f9a2-2ca0-4213-b3cc-3b6ffd105539 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:34.519311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 53/103] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:34.523144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9f9a2-2ca0-4213-b3cc-3b6ffd105539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-faa9f9a2-2ca0-4213-b3cc-3b6ffd105539 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:34.528020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:34.528859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:34.529595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:34.529595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:34.529595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:34.529595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:34.529595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:34.529595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:34.529595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:34.529595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:34.529595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:34.529595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:34.529595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:34.529595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:34.534915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 51/104] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:34.540519 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11691f-f381-4656-a192-fc9c416891a0 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:34.604997 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9f9a2-2ca0-4213-b3cc-3b6ffd105539 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:34.700264 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11691f-f381-4656-a192-fc9c416891a0 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:34.792596 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d0993-60e2-4c4d-9c78-992d3b0086aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:34.793237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d0993-60e2-4c4d-9c78-992d3b0086aa None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:34.793494 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d0993-60e2-4c4d-9c78-992d3b0086aa None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:34.793755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d0993-60e2-4c4d-9c78-992d3b0086aa None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:34.794438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d0993-60e2-4c4d-9c78-992d3b0086aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:34.794765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d0993-60e2-4c4d-9c78-992d3b0086aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:34.796170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d0993-60e2-4c4d-9c78-992d3b0086aa None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:34.801360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9f9a2-2ca0-4213-b3cc-3b6ffd105539 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:34.817381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 54/105] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:34.818445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:34.822770 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d0993-60e2-4c4d-9c78-992d3b0086aa None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:34.854829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c212e1a-3758-4366-b049-c28c210fab14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:34.855588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c212e1a-3758-4366-b049-c28c210fab14 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:34.855815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c212e1a-3758-4366-b049-c28c210fab14 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:34.856096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c212e1a-3758-4366-b049-c28c210fab14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:34.874785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c212e1a-3758-4366-b049-c28c210fab14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-6c212e1a-3758-4366-b049-c28c210fab14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:34.879805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:34.880624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:34.881836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:34.881836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:34.881836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:34.881836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:34.881836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:34.881836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:34.881836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:34.881836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:34.881836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:34.881836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:34.881836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:34.881836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:34.881836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 52/106] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:17:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:34.887109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c212e1a-3758-4366-b049-c28c210fab14 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:34.907677 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:34.982021 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d0993-60e2-4c4d-9c78-992d3b0086aa None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:35.010091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c212e1a-3758-4366-b049-c28c210fab14 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:35.018015 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5f2a1-0ec3-43ad-a326-fdb2e9ed8409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:35.018700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5f2a1-0ec3-43ad-a326-fdb2e9ed8409 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.018986 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5f2a1-0ec3-43ad-a326-fdb2e9ed8409 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.019261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5f2a1-0ec3-43ad-a326-fdb2e9ed8409 None admin] PATH_INFO: `/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.020654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5f2a1-0ec3-43ad-a326-fdb2e9ed8409 None admin] RBAC: Authorizing `identity:get_user(user_id=6ca6948ab5ab429ab41c690eaa6cc6c5)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:35.022218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5f2a1-0ec3-43ad-a326-fdb2e9ed8409 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:35.024001 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 55/107] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:34 2022] GET /identity/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:35.029410 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5f2a1-0ec3-43ad-a326-fdb2e9ed8409 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:35.045344 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9703b57-aa1d-4bad-9de9-c1144c6771ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:35.045965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9703b57-aa1d-4bad-9de9-c1144c6771ea None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.046243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9703b57-aa1d-4bad-9de9-c1144c6771ea None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.046528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9703b57-aa1d-4bad-9de9-c1144c6771ea None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.047206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9703b57-aa1d-4bad-9de9-c1144c6771ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:35.047507 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9703b57-aa1d-4bad-9de9-c1144c6771ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:35.048886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9703b57-aa1d-4bad-9de9-c1144c6771ea None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:35.057387 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 53/108] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:35.058414 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da467f-bec6-42b1-8d51-a8822fc74a65 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.058861 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da467f-bec6-42b1-8d51-a8822fc74a65 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.059145 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da467f-bec6-42b1-8d51-a8822fc74a65 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.060494 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 54/109] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:35.065039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da467f-bec6-42b1-8d51-a8822fc74a65 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:35.096687 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:35.139853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5f2a1-0ec3-43ad-a326-fdb2e9ed8409 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:35.143693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:35.175182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da467f-bec6-42b1-8d51-a8822fc74a65 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:35.177666 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450f507-759d-4813-8ec5-c072a3b0d17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:35.178389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450f507-759d-4813-8ec5-c072a3b0d17f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.178710 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450f507-759d-4813-8ec5-c072a3b0d17f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.178964 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450f507-759d-4813-8ec5-c072a3b0d17f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.190744 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450f507-759d-4813-8ec5-c072a3b0d17f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-c450f507-759d-4813-8ec5-c072a3b0d17f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:35.196735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:35.197574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:35.198305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:35.198305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:35.198305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:35.198305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:35.198305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:35.198305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:35.198305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:35.198305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:35.198305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:35.198305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:35.198305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:17:35.198305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:35.198933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 56/110] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:35.199869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cab37-759f-4135-a57c-1ac095aa337a None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.200157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cab37-759f-4135-a57c-1ac095aa337a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.200411 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cab37-759f-4135-a57c-1ac095aa337a None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.206126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 57/111] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:35.206126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cab37-759f-4135-a57c-1ac095aa337a None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:35.218793 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab203e4-da30-4d7c-9a01-c49822f6338b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:35.219798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab203e4-da30-4d7c-9a01-c49822f6338b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.220390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab203e4-da30-4d7c-9a01-c49822f6338b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.220717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab203e4-da30-4d7c-9a01-c49822f6338b None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.222178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab203e4-da30-4d7c-9a01-c49822f6338b None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:35.224760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab203e4-da30-4d7c-9a01-c49822f6338b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:35.232513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 55/112] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:35 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:35.232513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e127bd-adc3-4810-8a69-cf33e45fabc8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.232513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e127bd-adc3-4810-8a69-cf33e45fabc8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.232513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e127bd-adc3-4810-8a69-cf33e45fabc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.263611 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-74e127bd-adc3-4810-8a69-cf33e45fabc8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:35.264852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-74e127bd-adc3-4810-8a69-cf33e45fabc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:35.318376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cab37-759f-4135-a57c-1ac095aa337a None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:35.340590 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 56/113] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:35.341318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e127bd-adc3-4810-8a69-cf33e45fabc8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:35.360410 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01130689-6dd7-4548-b409-269ae787a049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:35.361062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01130689-6dd7-4548-b409-269ae787a049 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.361347 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01130689-6dd7-4548-b409-269ae787a049 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.361618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01130689-6dd7-4548-b409-269ae787a049 None admin] PATH_INFO: `/v3/users/94cbbd1503324af1af6df581aea570b9` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.362763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01130689-6dd7-4548-b409-269ae787a049 None admin] RBAC: Authorizing `identity:get_user(user_id=94cbbd1503324af1af6df581aea570b9)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:35.364237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01130689-6dd7-4548-b409-269ae787a049 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:35.366734 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 58/114] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:35 2022] GET /identity/v3/users/94cbbd1503324af1af6df581aea570b9 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:35.367133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd35e2-4053-44de-9f11-1d4097e4ad38 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.367417 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd35e2-4053-44de-9f11-1d4097e4ad38 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.367688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd35e2-4053-44de-9f11-1d4097e4ad38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.370963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:35.387674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-6bcd35e2-4053-44de-9f11-1d4097e4ad38 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:35.388863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-6bcd35e2-4053-44de-9f11-1d4097e4ad38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:35.414183 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:35.452167 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e127bd-adc3-4810-8a69-cf33e45fabc8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:35.458541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 59/115] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:35.458541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd35e2-4053-44de-9f11-1d4097e4ad38 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:35.493658 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3ade6-4ec0-4e13-bc98-704c7b19e6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:35.494624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3ade6-4ec0-4e13-bc98-704c7b19e6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.495104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3ade6-4ec0-4e13-bc98-704c7b19e6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.495585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3ade6-4ec0-4e13-bc98-704c7b19e6e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.496488 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3ade6-4ec0-4e13-bc98-704c7b19e6e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:35.496957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3ade6-4ec0-4e13-bc98-704c7b19e6e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:35.498575 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3ade6-4ec0-4e13-bc98-704c7b19e6e9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:35.507121 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 57/116] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:35.513326 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0aa7c1-ccc5-4a4e-99b8-d8aba5551fcf None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.513812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0aa7c1-ccc5-4a4e-99b8-d8aba5551fcf None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.514285 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0aa7c1-ccc5-4a4e-99b8-d8aba5551fcf None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.515750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 58/117] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:35.516718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0aa7c1-ccc5-4a4e-99b8-d8aba5551fcf None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:35.578994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd35e2-4053-44de-9f11-1d4097e4ad38 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:35.611303 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:35.618392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29968d6e-2ca2-4fe3-a3ae-8eb4eb887c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:35.619149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29968d6e-2ca2-4fe3-a3ae-8eb4eb887c26 None admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.619436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29968d6e-2ca2-4fe3-a3ae-8eb4eb887c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.619712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29968d6e-2ca2-4fe3-a3ae-8eb4eb887c26 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b/users/6ca6948ab5ab429ab41c690eaa6cc6c5/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.622127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29968d6e-2ca2-4fe3-a3ae-8eb4eb887c26 None admin] RBAC: Authorizing `identity:create_grant(project_id=c05ccc8ff85b456080a0930ce5f49c5b, user_id=6ca6948ab5ab429ab41c690eaa6cc6c5, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:35.623636 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29968d6e-2ca2-4fe3-a3ae-8eb4eb887c26 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:35.630441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0aa7c1-ccc5-4a4e-99b8-d8aba5551fcf None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:35.638457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 60/118] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:17:35 2022] PUT /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b/users/6ca6948ab5ab429ab41c690eaa6cc6c5/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:17:35.639110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29968d6e-2ca2-4fe3-a3ae-8eb4eb887c26 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:35.675376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:35.683530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c556ee-490e-4681-94c1-b5377bee22e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:35.684522 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c556ee-490e-4681-94c1-b5377bee22e6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.684934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c556ee-490e-4681-94c1-b5377bee22e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.685378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c556ee-490e-4681-94c1-b5377bee22e6 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.687182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c556ee-490e-4681-94c1-b5377bee22e6 None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:35.688945 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c556ee-490e-4681-94c1-b5377bee22e6 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:35.691002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 59/119] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:35 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:35.692854 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d6cbc-ce78-4995-b1b9-9548f12e4ab7 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.693296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d6cbc-ce78-4995-b1b9-9548f12e4ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.693701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d6cbc-ce78-4995-b1b9-9548f12e4ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.715749 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-915d6cbc-ce78-4995-b1b9-9548f12e4ab7 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:35.717476 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-915d6cbc-ce78-4995-b1b9-9548f12e4ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:35.765881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29968d6e-2ca2-4fe3-a3ae-8eb4eb887c26 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:35.798563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 60/120] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:35.798563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d6cbc-ce78-4995-b1b9-9548f12e4ab7 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:35.805159 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f9a2d-25a3-4ae4-8388-0f24259e50e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:35.806482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9a2d-25a3-4ae4-8388-0f24259e50e8 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.806922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9a2d-25a3-4ae4-8388-0f24259e50e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.807329 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9a2d-25a3-4ae4-8388-0f24259e50e8 None admin] PATH_INFO: `/v3/users/3b2028839dfb45e8947efd3bdbd9ab5f` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.826263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f9a2d-25a3-4ae4-8388-0f24259e50e8 None admin] RBAC: Authorizing `identity:get_user(user_id=3b2028839dfb45e8947efd3bdbd9ab5f)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:35.827467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f9a2d-25a3-4ae4-8388-0f24259e50e8 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:35.831147 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 61/121] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:35 2022] GET /identity/v3/users/3b2028839dfb45e8947efd3bdbd9ab5f => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:35.831147 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f9a2d-25a3-4ae4-8388-0f24259e50e8 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:35.914074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:35.918024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:35.921836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d6cbc-ce78-4995-b1b9-9548f12e4ab7 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:35.939324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f9a2d-25a3-4ae4-8388-0f24259e50e8 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:35.957177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a3498-57b0-410f-aad9-69e32865f842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:35.957964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a3498-57b0-410f-aad9-69e32865f842 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.958403 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a3498-57b0-410f-aad9-69e32865f842 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.958719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a3498-57b0-410f-aad9-69e32865f842 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.969397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a3498-57b0-410f-aad9-69e32865f842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-453a3498-57b0-410f-aad9-69e32865f842 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:35.974629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:35.975713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:35.976697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:35.976697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:35.976697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:35.976697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:35.976697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:35.976697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:35.976697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:35.976697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:35.976697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:35.976697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:35.976697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:17:35.976697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:35.978536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 61/122] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:17:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:35.984252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a3498-57b0-410f-aad9-69e32865f842 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:35.986532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2abe3-7724-484b-94ec-fa432d512e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:35.987269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2abe3-7724-484b-94ec-fa432d512e1b None admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:35.987551 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2abe3-7724-484b-94ec-fa432d512e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:35.987875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2abe3-7724-484b-94ec-fa432d512e1b None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b/users/94cbbd1503324af1af6df581aea570b9/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:35.991162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2abe3-7724-484b-94ec-fa432d512e1b None admin] RBAC: Authorizing `identity:create_grant(project_id=c05ccc8ff85b456080a0930ce5f49c5b, user_id=94cbbd1503324af1af6df581aea570b9, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:35.991363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2abe3-7724-484b-94ec-fa432d512e1b None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:36.009356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 62/123] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:17:35 2022] PUT /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b/users/94cbbd1503324af1af6df581aea570b9/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:17:36.009660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2abe3-7724-484b-94ec-fa432d512e1b None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:36.127467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2abe3-7724-484b-94ec-fa432d512e1b None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:36.131868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a3498-57b0-410f-aad9-69e32865f842 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:36.167391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2521561-e265-484d-8fb9-b1359caf9813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:36.168037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2521561-e265-484d-8fb9-b1359caf9813 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:36.168321 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2521561-e265-484d-8fb9-b1359caf9813 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:36.168594 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2521561-e265-484d-8fb9-b1359caf9813 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:36.181039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23197bbb-d5c1-4715-83f0-e7bc0d967099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:36.182347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23197bbb-d5c1-4715-83f0-e7bc0d967099 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:36.182967 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23197bbb-d5c1-4715-83f0-e7bc0d967099 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:36.183495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23197bbb-d5c1-4715-83f0-e7bc0d967099 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:36.185154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23197bbb-d5c1-4715-83f0-e7bc0d967099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:36.186915 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:36.187796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23197bbb-d5c1-4715-83f0-e7bc0d967099 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:36.194631 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 62/124] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:36.195733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23197bbb-d5c1-4715-83f0-e7bc0d967099 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:36.216024 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:36.257913 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2521561-e265-484d-8fb9-b1359caf9813 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:36.259424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2521561-e265-484d-8fb9-b1359caf9813 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:36.316621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23197bbb-d5c1-4715-83f0-e7bc0d967099 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:36.373761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d2aab-53a7-4383-80c1-6f233035ce61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:36.374482 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d2aab-53a7-4383-80c1-6f233035ce61 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:36.374851 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d2aab-53a7-4383-80c1-6f233035ce61 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:36.375190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d2aab-53a7-4383-80c1-6f233035ce61 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:36.375938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d2aab-53a7-4383-80c1-6f233035ce61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:36.376306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d2aab-53a7-4383-80c1-6f233035ce61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:36.377036 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 63/125] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:17:36.378374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2521561-e265-484d-8fb9-b1359caf9813 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:36.383219 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d2aab-53a7-4383-80c1-6f233035ce61 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:36.394983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 63/126] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:17:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:36.396065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d2aab-53a7-4383-80c1-6f233035ce61 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:36.427514 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:36.491957 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2521561-e265-484d-8fb9-b1359caf9813 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:36.502772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:36.534470 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab42bb-ca02-42fb-9731-84024767cfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:36.535166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab42bb-ca02-42fb-9731-84024767cfd1 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:36.535453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab42bb-ca02-42fb-9731-84024767cfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:36.535733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab42bb-ca02-42fb-9731-84024767cfd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:36.540728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d2aab-53a7-4383-80c1-6f233035ce61 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:36.547848 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab42bb-ca02-42fb-9731-84024767cfd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-c9ab42bb-ca02-42fb-9731-84024767cfd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:36.553265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:36.554106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:36.554930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:36.554930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:36.554930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:36.554930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:36.554930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:36.554930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:36.554930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:36.554930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:36.554930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:36.554930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:36.554930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:36.554930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:36.554930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 64/127] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:36.559992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab42bb-ca02-42fb-9731-84024767cfd1 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:36.585505 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82c1f8-e022-4d6c-be0f-22237bb960f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:36.586131 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82c1f8-e022-4d6c-be0f-22237bb960f7 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:36.586414 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82c1f8-e022-4d6c-be0f-22237bb960f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:36.586741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82c1f8-e022-4d6c-be0f-22237bb960f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:36.587940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82c1f8-e022-4d6c-be0f-22237bb960f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:36.589956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82c1f8-e022-4d6c-be0f-22237bb960f7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:36.627598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 64/128] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Sep 9 16:17:36 2022] POST /identity/v3/projects => generated 302 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:17:36.631465 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82c1f8-e022-4d6c-be0f-22237bb960f7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:36.762070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:36.768841 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab42bb-ca02-42fb-9731-84024767cfd1 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:36.790153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82c1f8-e022-4d6c-be0f-22237bb960f7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:36.848523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:36.860165 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f6de7-689b-4d0e-9cf0-09124d504b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:36.860819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f6de7-689b-4d0e-9cf0-09124d504b54 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:36.861102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f6de7-689b-4d0e-9cf0-09124d504b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:36.861410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f6de7-689b-4d0e-9cf0-09124d504b54 None admin] PATH_INFO: `/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:36.863087 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e51863-4f84-46ff-a43d-074786eec304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:36.863638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f6de7-689b-4d0e-9cf0-09124d504b54 None admin] RBAC: Authorizing `identity:get_user(user_id=6ca6948ab5ab429ab41c690eaa6cc6c5)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:36.863790 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e51863-4f84-46ff-a43d-074786eec304 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:36.864021 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e51863-4f84-46ff-a43d-074786eec304 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:36.864298 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e51863-4f84-46ff-a43d-074786eec304 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:36.864896 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f6de7-689b-4d0e-9cf0-09124d504b54 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:36.865452 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e51863-4f84-46ff-a43d-074786eec304 None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:36.866525 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 65/129] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:36 2022] GET /identity/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:36.867351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f6de7-689b-4d0e-9cf0-09124d504b54 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:36.869611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e51863-4f84-46ff-a43d-074786eec304 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:36.875241 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 65/130] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:36 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:36.876542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e51863-4f84-46ff-a43d-074786eec304 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:36.980901 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e51863-4f84-46ff-a43d-074786eec304 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:36.983666 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f6de7-689b-4d0e-9cf0-09124d504b54 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:37.017304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca6e19-8f8b-4800-bea7-1300e9405fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:37.017896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca6e19-8f8b-4800-bea7-1300e9405fef None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:37.018183 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca6e19-8f8b-4800-bea7-1300e9405fef None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:37.018463 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca6e19-8f8b-4800-bea7-1300e9405fef None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:37.021092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca6e19-8f8b-4800-bea7-1300e9405fef None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:37.022914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca6e19-8f8b-4800-bea7-1300e9405fef None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:37.029773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 66/131] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:36 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:37.030951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca6e19-8f8b-4800-bea7-1300e9405fef None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:37.032229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0921fa-0b0f-4aa2-9c07-3fdd9428c8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:37.032869 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0921fa-0b0f-4aa2-9c07-3fdd9428c8bd None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:37.033146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0921fa-0b0f-4aa2-9c07-3fdd9428c8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:37.033419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0921fa-0b0f-4aa2-9c07-3fdd9428c8bd None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:37.034159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0921fa-0b0f-4aa2-9c07-3fdd9428c8bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:37.034628 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0921fa-0b0f-4aa2-9c07-3fdd9428c8bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:37.036062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0921fa-0b0f-4aa2-9c07-3fdd9428c8bd None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:37.051372 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 66/132] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:37.051673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0921fa-0b0f-4aa2-9c07-3fdd9428c8bd None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:37.083571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:37.128757 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:37.149569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca6e19-8f8b-4800-bea7-1300e9405fef None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:37.171757 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0921fa-0b0f-4aa2-9c07-3fdd9428c8bd None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:37.185581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3bf67-a126-4833-9a5d-91e52d1bdcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:37.186525 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3bf67-a126-4833-9a5d-91e52d1bdcb0 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:37.187003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3bf67-a126-4833-9a5d-91e52d1bdcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:37.187469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3bf67-a126-4833-9a5d-91e52d1bdcb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:37.188530 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3bf67-a126-4833-9a5d-91e52d1bdcb0 None admin] RBAC: Adding query filter params (role.id=17a32575480045789f8f279a3ada4fd1, scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b, user.id=3b2028839dfb45e8947efd3bdbd9ab5f) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:37.189059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3bf67-a126-4833-9a5d-91e52d1bdcb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:37.190747 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3bf67-a126-4833-9a5d-91e52d1bdcb0 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:37.197188 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 67/133] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:37 2022] GET /identity/v3/role_assignments?user.id=3b2028839dfb45e8947efd3bdbd9ab5f&scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b&role.id=17a32575480045789f8f279a3ada4fd1 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:37.198437 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3bf67-a126-4833-9a5d-91e52d1bdcb0 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:37.213808 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e068f-2f14-4fcc-bb30-8bd00c3163a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:37.214604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e068f-2f14-4fcc-bb30-8bd00c3163a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:37.214889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e068f-2f14-4fcc-bb30-8bd00c3163a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:37.215164 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e068f-2f14-4fcc-bb30-8bd00c3163a9 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b/users/6ca6948ab5ab429ab41c690eaa6cc6c5/roles/0d40b3176f78460f9c4a29c1e9ef843e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:37.219329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e068f-2f14-4fcc-bb30-8bd00c3163a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c05ccc8ff85b456080a0930ce5f49c5b, user_id=6ca6948ab5ab429ab41c690eaa6cc6c5, role_id=0d40b3176f78460f9c4a29c1e9ef843e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:37.220919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e068f-2f14-4fcc-bb30-8bd00c3163a9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:37.251209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 67/134] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:17:37 2022] PUT /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b/users/6ca6948ab5ab429ab41c690eaa6cc6c5/roles/0d40b3176f78460f9c4a29c1e9ef843e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:17:37.356559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:37.361038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3bf67-a126-4833-9a5d-91e52d1bdcb0 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:37.401967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6fbe3-be01-4c21-b932-2ac1b9a065bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:37.402673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6fbe3-be01-4c21-b932-2ac1b9a065bb None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:37.402960 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6fbe3-be01-4c21-b932-2ac1b9a065bb None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:37.403236 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6fbe3-be01-4c21-b932-2ac1b9a065bb None admin] PATH_INFO: `/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:37.404637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6fbe3-be01-4c21-b932-2ac1b9a065bb None admin] RBAC: Authorizing `identity:get_user(user_id=6ca6948ab5ab429ab41c690eaa6cc6c5)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:37.406187 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6fbe3-be01-4c21-b932-2ac1b9a065bb None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:37.408209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 68/135] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:37 2022] GET /identity/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:37.414213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e068f-2f14-4fcc-bb30-8bd00c3163a9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:37.523842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e068f-2f14-4fcc-bb30-8bd00c3163a9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:37.546940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:37.562139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac8851-cf81-4743-b2d2-d82ebb906871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:37.562849 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac8851-cf81-4743-b2d2-d82ebb906871 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:37.563123 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac8851-cf81-4743-b2d2-d82ebb906871 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:37.563554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac8851-cf81-4743-b2d2-d82ebb906871 None admin] PATH_INFO: `/v3/projects/a2abfc5ca6da4956ac25b8040e6b4e58` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:37.564812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac8851-cf81-4743-b2d2-d82ebb906871 None admin] RBAC: Authorizing `identity:get_project(project_id=a2abfc5ca6da4956ac25b8040e6b4e58)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:37.566276 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac8851-cf81-4743-b2d2-d82ebb906871 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:37.572458 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 68/136] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:37 2022] GET /identity/v3/projects/a2abfc5ca6da4956ac25b8040e6b4e58 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:37.575191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6fbe3-be01-4c21-b932-2ac1b9a065bb None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:37.684155 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6fbe3-be01-4c21-b932-2ac1b9a065bb None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:37.721664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da7396-fd44-4b54-89c8-79a087d4fce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:37.722336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da7396-fd44-4b54-89c8-79a087d4fce3 None admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:37.722645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da7396-fd44-4b54-89c8-79a087d4fce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:37.722919 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da7396-fd44-4b54-89c8-79a087d4fce3 None admin] PATH_INFO: `/v3/projects/a2abfc5ca6da4956ac25b8040e6b4e58/users/6ca6948ab5ab429ab41c690eaa6cc6c5/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:37.724831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da7396-fd44-4b54-89c8-79a087d4fce3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2abfc5ca6da4956ac25b8040e6b4e58, user_id=6ca6948ab5ab429ab41c690eaa6cc6c5, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:37.726645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da7396-fd44-4b54-89c8-79a087d4fce3 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:37.742775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 69/137] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:17:37 2022] PUT /identity/v3/projects/a2abfc5ca6da4956ac25b8040e6b4e58/users/6ca6948ab5ab429ab41c690eaa6cc6c5/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:17:38.669150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5366d-9736-4032-9186-8a2bd7dd8032 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:38.669857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5366d-9736-4032-9186-8a2bd7dd8032 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:38.669857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5366d-9736-4032-9186-8a2bd7dd8032 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:38.671298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 69/138] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:38.676856 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a2dee-f91d-42a9-b510-51bec4637411 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:38.677342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a2dee-f91d-42a9-b510-51bec4637411 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:38.677491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a2dee-f91d-42a9-b510-51bec4637411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:38.696502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-105a2dee-f91d-42a9-b510-51bec4637411 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:38.697773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-105a2dee-f91d-42a9-b510-51bec4637411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:38.721262 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:38.771606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 70/139] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:17:39.008674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb5e2e-45f2-4e67-9466-7346667983d5 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:39.009007 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb5e2e-45f2-4e67-9466-7346667983d5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:39.009171 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb5e2e-45f2-4e67-9466-7346667983d5 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:39.010697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 70/140] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:39.016150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b9b3c-7fcb-463d-81f3-fb125913c798 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:39.016437 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b9b3c-7fcb-463d-81f3-fb125913c798 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:39.016704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b9b3c-7fcb-463d-81f3-fb125913c798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:39.044591 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-681b9b3c-7fcb-463d-81f3-fb125913c798 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:39.045778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-681b9b3c-7fcb-463d-81f3-fb125913c798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:39.141390 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 71/141] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:39.145066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb5e2e-45f2-4e67-9466-7346667983d5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:39.248678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb5e2e-45f2-4e67-9466-7346667983d5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:39.283422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be80549-25f8-42b2-966e-1884b53a7c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:39.283976 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be80549-25f8-42b2-966e-1884b53a7c3e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:39.284253 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be80549-25f8-42b2-966e-1884b53a7c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:39.284521 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be80549-25f8-42b2-966e-1884b53a7c3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:39.295125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be80549-25f8-42b2-966e-1884b53a7c3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-7be80549-25f8-42b2-966e-1884b53a7c3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:39.300014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:39.300821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:39.301544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:39.301544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:39.301544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:39.301544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:39.301544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:39.301544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:39.301544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:39.301544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:39.301544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:39.301544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:39.301544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:39.301544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:39.301544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 71/142] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:17:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:17:39.308263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b9b3c-7fcb-463d-81f3-fb125913c798 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:39.397988 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:39.410309 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b9b3c-7fcb-463d-81f3-fb125913c798 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:39.446096 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08ae02-a6e7-431f-98f1-e8db46cc7572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:39.446826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ae02-a6e7-431f-98f1-e8db46cc7572 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:39.447103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ae02-a6e7-431f-98f1-e8db46cc7572 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:39.447644 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ae02-a6e7-431f-98f1-e8db46cc7572 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:39.448387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08ae02-a6e7-431f-98f1-e8db46cc7572 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:39.448689 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08ae02-a6e7-431f-98f1-e8db46cc7572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:39.450139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08ae02-a6e7-431f-98f1-e8db46cc7572 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:39.459719 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 72/143] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:39.465654 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be80549-25f8-42b2-966e-1884b53a7c3e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:39.515682 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:39.569930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be80549-25f8-42b2-966e-1884b53a7c3e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:39.605629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56380b-8d19-46cc-a1dc-62a2ea174973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:39.606265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56380b-8d19-46cc-a1dc-62a2ea174973 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:39.606563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56380b-8d19-46cc-a1dc-62a2ea174973 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:39.606837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56380b-8d19-46cc-a1dc-62a2ea174973 None admin] PATH_INFO: `/v3/users/94cbbd1503324af1af6df581aea570b9` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:39.608025 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56380b-8d19-46cc-a1dc-62a2ea174973 None admin] RBAC: Authorizing `identity:get_user(user_id=94cbbd1503324af1af6df581aea570b9)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:39.609478 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56380b-8d19-46cc-a1dc-62a2ea174973 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:39.611076 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 72/144] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:39 2022] GET /identity/v3/users/94cbbd1503324af1af6df581aea570b9 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:39.616896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08ae02-a6e7-431f-98f1-e8db46cc7572 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:39.689879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56380b-8d19-46cc-a1dc-62a2ea174973 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:39.761855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08ae02-a6e7-431f-98f1-e8db46cc7572 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:39.834579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d853c56-1ad7-40b4-859b-0266b371f984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:39.835187 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d853c56-1ad7-40b4-859b-0266b371f984 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:39.835544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d853c56-1ad7-40b4-859b-0266b371f984 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:39.835813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d853c56-1ad7-40b4-859b-0266b371f984 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:39.886912 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:39.892681 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56380b-8d19-46cc-a1dc-62a2ea174973 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:39.938935 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:39.969075 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d853c56-1ad7-40b4-859b-0266b371f984 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:39.970591 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d853c56-1ad7-40b4-859b-0266b371f984 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:39.974295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf1d5a-0ee8-49a3-9eba-7c0c0eb79a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:39.975200 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf1d5a-0ee8-49a3-9eba-7c0c0eb79a34 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:39.975676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf1d5a-0ee8-49a3-9eba-7c0c0eb79a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:39.976276 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf1d5a-0ee8-49a3-9eba-7c0c0eb79a34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:39.999549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf1d5a-0ee8-49a3-9eba-7c0c0eb79a34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-92cf1d5a-0ee8-49a3-9eba-7c0c0eb79a34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:40.006348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:40.007390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:40.008150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:40.008150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:40.008150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:40.008150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:40.008150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:40.008150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:40.008150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:40.008150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:40.008150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:40.008150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:40.008150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:40.008150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:40.018533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 73/145] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:40.020385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf1d5a-0ee8-49a3-9eba-7c0c0eb79a34 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:40.089985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 73/146] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 16:17:40.095381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d853c56-1ad7-40b4-859b-0266b371f984 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:40.199469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf1d5a-0ee8-49a3-9eba-7c0c0eb79a34 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:40.222389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:40.242153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09346a0a-5dc9-4974-bbd5-1c3dac84e204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:40.243314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09346a0a-5dc9-4974-bbd5-1c3dac84e204 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:40.243602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09346a0a-5dc9-4974-bbd5-1c3dac84e204 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:40.243877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09346a0a-5dc9-4974-bbd5-1c3dac84e204 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:40.244694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09346a0a-5dc9-4974-bbd5-1c3dac84e204 None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:40.244999 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09346a0a-5dc9-4974-bbd5-1c3dac84e204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:40.246676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09346a0a-5dc9-4974-bbd5-1c3dac84e204 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:40.257693 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 74/147] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:40.258980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec476569-570b-4641-9e10-d385450b14c0 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:40.259270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec476569-570b-4641-9e10-d385450b14c0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:40.259543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec476569-570b-4641-9e10-d385450b14c0 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:40.261427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 75/148] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:40.266788 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec476569-570b-4641-9e10-d385450b14c0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:40.302801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d853c56-1ad7-40b4-859b-0266b371f984 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:40.313257 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:40.347838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933a47a-5364-4867-be84-d05f17eb80f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:40.348906 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933a47a-5364-4867-be84-d05f17eb80f6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:40.349435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933a47a-5364-4867-be84-d05f17eb80f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:40.349842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933a47a-5364-4867-be84-d05f17eb80f6 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:40.351284 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933a47a-5364-4867-be84-d05f17eb80f6 None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:40.354096 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933a47a-5364-4867-be84-d05f17eb80f6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:40.356225 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 74/149] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:40 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:40.363837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bceb7a-0af1-4571-9a92-30ab6ce5a22e None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:40.364360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bceb7a-0af1-4571-9a92-30ab6ce5a22e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:40.364823 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bceb7a-0af1-4571-9a92-30ab6ce5a22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:40.387885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-e4bceb7a-0af1-4571-9a92-30ab6ce5a22e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:40.389458 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e4bceb7a-0af1-4571-9a92-30ab6ce5a22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:40.399185 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec476569-570b-4641-9e10-d385450b14c0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:40.440915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a7c4e-5f6d-432b-8b36-ac435b4f9651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:40.442561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a7c4e-5f6d-432b-8b36-ac435b4f9651 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:40.442955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a7c4e-5f6d-432b-8b36-ac435b4f9651 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:40.443325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a7c4e-5f6d-432b-8b36-ac435b4f9651 None admin] PATH_INFO: `/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:40.444992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a7c4e-5f6d-432b-8b36-ac435b4f9651 None admin] RBAC: Authorizing `identity:get_user(user_id=6ca6948ab5ab429ab41c690eaa6cc6c5)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:40.447261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a7c4e-5f6d-432b-8b36-ac435b4f9651 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:40.449177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 76/150] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:40 2022] GET /identity/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:40.451389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c92be-d87b-454d-a194-ad6021cea896 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:40.451775 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c92be-d87b-454d-a194-ad6021cea896 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:40.452176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c92be-d87b-454d-a194-ad6021cea896 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:40.453992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 77/151] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:40.454754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c92be-d87b-454d-a194-ad6021cea896 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:40.472460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 75/152] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:40.473586 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe54b7-e96a-4a8a-8cd8-8d47a46f0735 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:40.473869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe54b7-e96a-4a8a-8cd8-8d47a46f0735 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:40.474133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe54b7-e96a-4a8a-8cd8-8d47a46f0735 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:40.475640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 76/153] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:40.476282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe54b7-e96a-4a8a-8cd8-8d47a46f0735 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:40.533767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:40.577502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c92be-d87b-454d-a194-ad6021cea896 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:40.588149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:40.590710 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe54b7-e96a-4a8a-8cd8-8d47a46f0735 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:40.623310 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266d3ff-cca4-4c8a-8edc-2d8ba73cff0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:40.624453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266d3ff-cca4-4c8a-8edc-2d8ba73cff0b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:40.624929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266d3ff-cca4-4c8a-8edc-2d8ba73cff0b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:40.625593 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266d3ff-cca4-4c8a-8edc-2d8ba73cff0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:40.626939 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266d3ff-cca4-4c8a-8edc-2d8ba73cff0b None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b, user.id=94cbbd1503324af1af6df581aea570b9) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:40.627678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266d3ff-cca4-4c8a-8edc-2d8ba73cff0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:40.629436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266d3ff-cca4-4c8a-8edc-2d8ba73cff0b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:40.631701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2423fbf-a8f0-42d7-ae7e-e0b695a360f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:40.632398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2423fbf-a8f0-42d7-ae7e-e0b695a360f9 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:40.632745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2423fbf-a8f0-42d7-ae7e-e0b695a360f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:40.633083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2423fbf-a8f0-42d7-ae7e-e0b695a360f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:40.637763 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 78/154] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:40 2022] GET /identity/v3/role_assignments?user.id=94cbbd1503324af1af6df581aea570b9&scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b&role.id=c6da7ba2be504943af29338ee8855f05 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:40.639364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6fa48-a5cf-4ce8-bc02-78989721457b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:40.639818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6fa48-a5cf-4ce8-bc02-78989721457b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:40.640253 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6fa48-a5cf-4ce8-bc02-78989721457b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:40.679619 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b0f6fa48-a5cf-4ce8-bc02-78989721457b None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:40.683660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b0f6fa48-a5cf-4ce8-bc02-78989721457b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:40.755720 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2423fbf-a8f0-42d7-ae7e-e0b695a360f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:40.757847 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2423fbf-a8f0-42d7-ae7e-e0b695a360f9 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:40.811118 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:40.831331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 79/155] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:40.839995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b22e5-7a77-40e9-b65a-74fc0a62109d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:40.840275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b22e5-7a77-40e9-b65a-74fc0a62109d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:40.840547 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b22e5-7a77-40e9-b65a-74fc0a62109d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:40.874683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c79b22e5-7a77-40e9-b65a-74fc0a62109d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:40.874842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 77/156] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:17:40.874931 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c79b22e5-7a77-40e9-b65a-74fc0a62109d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:40.875000 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2423fbf-a8f0-42d7-ae7e-e0b695a360f9 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:40.971090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 80/157] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:40.972604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b22e5-7a77-40e9-b65a-74fc0a62109d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:41.007093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2423fbf-a8f0-42d7-ae7e-e0b695a360f9 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:41.021973 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:41.081295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63befe-d85d-4005-b272-0ffe1000dcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:41.083282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63befe-d85d-4005-b272-0ffe1000dcfa None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:41.083559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63befe-d85d-4005-b272-0ffe1000dcfa None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:41.083814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63befe-d85d-4005-b272-0ffe1000dcfa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:41.113595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63befe-d85d-4005-b272-0ffe1000dcfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-ef63befe-d85d-4005-b272-0ffe1000dcfa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:41.116270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:41.117273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:41.118409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:41.118409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:41.118409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:41.118409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:41.118409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:41.118409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:41.118409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:41.118409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:41.118409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:41.118409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:41.118409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:17:41.118409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:41.118409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 78/158] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:41.119367 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb0a61-78b6-49fd-bfff-c00a7b0631e7 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:41.119367 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb0a61-78b6-49fd-bfff-c00a7b0631e7 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:41.119367 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb0a61-78b6-49fd-bfff-c00a7b0631e7 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:41.121716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 79/159] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:41.126751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb0a61-78b6-49fd-bfff-c00a7b0631e7 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:41.142090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:41.161149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b22e5-7a77-40e9-b65a-74fc0a62109d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:41.195588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40b03a-ea25-4e56-adc5-e4ee1d7c4a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:41.196231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40b03a-ea25-4e56-adc5-e4ee1d7c4a48 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:41.196507 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40b03a-ea25-4e56-adc5-e4ee1d7c4a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:41.196777 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40b03a-ea25-4e56-adc5-e4ee1d7c4a48 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:41.197904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40b03a-ea25-4e56-adc5-e4ee1d7c4a48 None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:41.199417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40b03a-ea25-4e56-adc5-e4ee1d7c4a48 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:41.201288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 81/160] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:40 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:41.202200 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30899f23-ef7f-4795-b854-b2f2151c253c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:41.202474 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30899f23-ef7f-4795-b854-b2f2151c253c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:41.202764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30899f23-ef7f-4795-b854-b2f2151c253c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:41.224002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-30899f23-ef7f-4795-b854-b2f2151c253c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:41.225198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-30899f23-ef7f-4795-b854-b2f2151c253c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:41.248078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb0a61-78b6-49fd-bfff-c00a7b0631e7 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:41.283804 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe73e1c-3851-4b14-a784-c6e5fb1ebe42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:41.284424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73e1c-3851-4b14-a784-c6e5fb1ebe42 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:41.284701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73e1c-3851-4b14-a784-c6e5fb1ebe42 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:41.284975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73e1c-3851-4b14-a784-c6e5fb1ebe42 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:41.285642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe73e1c-3851-4b14-a784-c6e5fb1ebe42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:41.285943 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe73e1c-3851-4b14-a784-c6e5fb1ebe42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:41.287373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe73e1c-3851-4b14-a784-c6e5fb1ebe42 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:41.293772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 82/161] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:41.294643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30899f23-ef7f-4795-b854-b2f2151c253c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:41.296219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 80/162] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:41.297026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe73e1c-3851-4b14-a784-c6e5fb1ebe42 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:41.327715 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:41.373212 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:41.403217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30899f23-ef7f-4795-b854-b2f2151c253c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:41.406954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe73e1c-3851-4b14-a784-c6e5fb1ebe42 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:41.439949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966f03d-39d6-43a6-a448-2205ef1ec8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:41.440647 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966f03d-39d6-43a6-a448-2205ef1ec8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:41.440927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966f03d-39d6-43a6-a448-2205ef1ec8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:41.441251 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966f03d-39d6-43a6-a448-2205ef1ec8f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:41.442352 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966f03d-39d6-43a6-a448-2205ef1ec8f6 None admin] RBAC: Adding query filter params (role.id=9e02fa627e5c4388b0aa7182f1707895, scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b, user.id=6ca6948ab5ab429ab41c690eaa6cc6c5) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:41.442694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966f03d-39d6-43a6-a448-2205ef1ec8f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:41.444161 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966f03d-39d6-43a6-a448-2205ef1ec8f6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:41.444367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982f831-72bd-4866-ae04-53e2ff7fe1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:41.445056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982f831-72bd-4866-ae04-53e2ff7fe1bc None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:41.445325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982f831-72bd-4866-ae04-53e2ff7fe1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:41.445585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982f831-72bd-4866-ae04-53e2ff7fe1bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:41.446692 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982f831-72bd-4866-ae04-53e2ff7fe1bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:41.448153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982f831-72bd-4866-ae04-53e2ff7fe1bc None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:41.454571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 83/163] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:41.454571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982f831-72bd-4866-ae04-53e2ff7fe1bc None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:41.454802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 81/164] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:41 2022] GET /identity/v3/role_assignments?user.id=6ca6948ab5ab429ab41c690eaa6cc6c5&scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b&role.id=9e02fa627e5c4388b0aa7182f1707895 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:41.454802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966f03d-39d6-43a6-a448-2205ef1ec8f6 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:41.561181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:41.569040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982f831-72bd-4866-ae04-53e2ff7fe1bc None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:41.594917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966f03d-39d6-43a6-a448-2205ef1ec8f6 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:41.609152 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ff7fe-8046-40f7-9b97-63a71b1dd2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:41.609813 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ff7fe-8046-40f7-9b97-63a71b1dd2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:41.610079 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ff7fe-8046-40f7-9b97-63a71b1dd2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:41.610346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ff7fe-8046-40f7-9b97-63a71b1dd2b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:41.622380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ff7fe-8046-40f7-9b97-63a71b1dd2b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-7a9ff7fe-8046-40f7-9b97-63a71b1dd2b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:41.627863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:41.628728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:41.631425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:41.631425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:41.631425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:41.631425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:41.631425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:41.631425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:41.631425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:41.631425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:41.631425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:41.631425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:41.631425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:17:41.631425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:41.631425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 84/165] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:17:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:41.631425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ff7fe-8046-40f7-9b97-63a71b1dd2b7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:41.636973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bf8a0-aa90-4ad6-8010-5eab59f70469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:41.637882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bf8a0-aa90-4ad6-8010-5eab59f70469 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:41.638158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bf8a0-aa90-4ad6-8010-5eab59f70469 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:41.638426 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bf8a0-aa90-4ad6-8010-5eab59f70469 None admin] PATH_INFO: `/v3/users/3b2028839dfb45e8947efd3bdbd9ab5f` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:41.639992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bf8a0-aa90-4ad6-8010-5eab59f70469 None admin] RBAC: Authorizing `identity:get_user(user_id=3b2028839dfb45e8947efd3bdbd9ab5f)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:41.641749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bf8a0-aa90-4ad6-8010-5eab59f70469 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:41.643314 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 82/166] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:41 2022] GET /identity/v3/users/3b2028839dfb45e8947efd3bdbd9ab5f => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:41.643511 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bf8a0-aa90-4ad6-8010-5eab59f70469 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:41.651108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:41.741705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ff7fe-8046-40f7-9b97-63a71b1dd2b7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:41.756535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bf8a0-aa90-4ad6-8010-5eab59f70469 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:41.776528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e35f03-fec0-408b-8fe7-444b276710fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:41.777146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e35f03-fec0-408b-8fe7-444b276710fe None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:41.777428 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e35f03-fec0-408b-8fe7-444b276710fe None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:41.777703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e35f03-fec0-408b-8fe7-444b276710fe None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:41.778351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e35f03-fec0-408b-8fe7-444b276710fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:41.779864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e35f03-fec0-408b-8fe7-444b276710fe None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:41.790392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5df1c8-b52f-4f5e-906f-16669a8e6612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:41.791172 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df1c8-b52f-4f5e-906f-16669a8e6612 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:41.791332 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df1c8-b52f-4f5e-906f-16669a8e6612 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:41.791606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df1c8-b52f-4f5e-906f-16669a8e6612 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:41.801974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 85/167] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:17:41 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:17:41.802294 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5df1c8-b52f-4f5e-906f-16669a8e6612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:41.802819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e35f03-fec0-408b-8fe7-444b276710fe None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-ab5df1c8-b52f-4f5e-906f-16669a8e6612 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:41.808054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:41.809134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:41.811207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:41.811207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:41.811207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:41.811207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:41.811207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:41.811207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:41.811207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:41.811207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:41.811207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:41.811207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:41.811207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:41.811207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:41.811207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 83/168] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:41.811207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5df1c8-b52f-4f5e-906f-16669a8e6612 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:41.819863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:41.886094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:41.930148 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e35f03-fec0-408b-8fe7-444b276710fe None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:41.937650 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5df1c8-b52f-4f5e-906f-16669a8e6612 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:41.970066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c0ddc-7c51-4062-af59-262e7092ce93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:41.970842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c0ddc-7c51-4062-af59-262e7092ce93 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:41.971121 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c0ddc-7c51-4062-af59-262e7092ce93 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:41.971376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c0ddc-7c51-4062-af59-262e7092ce93 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:41.972070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c0ddc-7c51-4062-af59-262e7092ce93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:41.972360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c0ddc-7c51-4062-af59-262e7092ce93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:41.973768 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c0ddc-7c51-4062-af59-262e7092ce93 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:41.983029 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 86/169] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:17:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:41.984155 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c0ddc-7c51-4062-af59-262e7092ce93 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:41.987207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a88e9-c44e-444f-9528-37dc71d925dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:41.987456 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a88e9-c44e-444f-9528-37dc71d925dd None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:41.987942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a88e9-c44e-444f-9528-37dc71d925dd None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:41.988110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a88e9-c44e-444f-9528-37dc71d925dd None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:41.989334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a88e9-c44e-444f-9528-37dc71d925dd None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:41.991028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a88e9-c44e-444f-9528-37dc71d925dd None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:41.993023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 84/170] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:41 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:41.998994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a88e9-c44e-444f-9528-37dc71d925dd None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:42.059609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:42.108340 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c0ddc-7c51-4062-af59-262e7092ce93 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:42.113734 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a88e9-c44e-444f-9528-37dc71d925dd None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:42.152461 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a1ed1-21c8-4cc8-80f3-553ae46adcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:42.153053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a1ed1-21c8-4cc8-80f3-553ae46adcdb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:42.153195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:42.153364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a1ed1-21c8-4cc8-80f3-553ae46adcdb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:42.153583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a1ed1-21c8-4cc8-80f3-553ae46adcdb None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:42.154260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a1ed1-21c8-4cc8-80f3-553ae46adcdb None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:42.154591 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a1ed1-21c8-4cc8-80f3-553ae46adcdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:42.155995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a1ed1-21c8-4cc8-80f3-553ae46adcdb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:42.161451 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b894a3-6534-42ce-a32e-e94ffd6fce9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:42.162684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b894a3-6534-42ce-a32e-e94ffd6fce9e None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:42.162885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b894a3-6534-42ce-a32e-e94ffd6fce9e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:42.163157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b894a3-6534-42ce-a32e-e94ffd6fce9e None admin] PATH_INFO: `/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:42.164676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b894a3-6534-42ce-a32e-e94ffd6fce9e None admin] RBAC: Authorizing `identity:get_user(user_id=6ca6948ab5ab429ab41c690eaa6cc6c5)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:42.165590 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 87/171] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:42.166232 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a1ed1-21c8-4cc8-80f3-553ae46adcdb None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:42.166336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b894a3-6534-42ce-a32e-e94ffd6fce9e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:42.168310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 85/172] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:41 2022] GET /identity/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:42.172660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b894a3-6534-42ce-a32e-e94ffd6fce9e None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:42.291293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a1ed1-21c8-4cc8-80f3-553ae46adcdb None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:42.292561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b894a3-6534-42ce-a32e-e94ffd6fce9e None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:42.331192 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d00922-39d8-429b-8e9d-d7b28d4b6d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:42.331882 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d00922-39d8-429b-8e9d-d7b28d4b6d4f None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:42.332158 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d00922-39d8-429b-8e9d-d7b28d4b6d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:42.332428 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d00922-39d8-429b-8e9d-d7b28d4b6d4f None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b/users/3b2028839dfb45e8947efd3bdbd9ab5f/roles/17a32575480045789f8f279a3ada4fd1` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:42.335044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d00922-39d8-429b-8e9d-d7b28d4b6d4f None admin] RBAC: Authorizing `identity:create_grant(project_id=c05ccc8ff85b456080a0930ce5f49c5b, user_id=3b2028839dfb45e8947efd3bdbd9ab5f, role_id=17a32575480045789f8f279a3ada4fd1)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:42.335758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e0a0c-a15a-48cc-9643-636d8734d46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:42.336406 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e0a0c-a15a-48cc-9643-636d8734d46c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:42.336569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d00922-39d8-429b-8e9d-d7b28d4b6d4f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:42.336666 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e0a0c-a15a-48cc-9643-636d8734d46c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:42.336940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e0a0c-a15a-48cc-9643-636d8734d46c None admin] PATH_INFO: `/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:42.338907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e0a0c-a15a-48cc-9643-636d8734d46c None admin] RBAC: Authorizing `identity:get_user(user_id=6ca6948ab5ab429ab41c690eaa6cc6c5)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:42.339374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:42.340200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e0a0c-a15a-48cc-9643-636d8734d46c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:42.342202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 86/173] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:42 2022] GET /identity/v3/users/6ca6948ab5ab429ab41c690eaa6cc6c5 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:42.343343 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e0a0c-a15a-48cc-9643-636d8734d46c None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:42.359394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 88/174] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:17:42 2022] PUT /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b/users/3b2028839dfb45e8947efd3bdbd9ab5f/roles/17a32575480045789f8f279a3ada4fd1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:17:42.360245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d00922-39d8-429b-8e9d-d7b28d4b6d4f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:42.459969 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:42.483571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d00922-39d8-429b-8e9d-d7b28d4b6d4f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:42.510752 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e0a0c-a15a-48cc-9643-636d8734d46c None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:42.555223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24859d-2bad-4439-a30e-08ea5f33f96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:42.555815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24859d-2bad-4439-a30e-08ea5f33f96f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:42.556094 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24859d-2bad-4439-a30e-08ea5f33f96f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:42.556364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24859d-2bad-4439-a30e-08ea5f33f96f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:42.578566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278caec5-eebd-4d61-a9db-1d059b9679c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:42.579204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278caec5-eebd-4d61-a9db-1d059b9679c9 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:42.579482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278caec5-eebd-4d61-a9db-1d059b9679c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:42.579749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278caec5-eebd-4d61-a9db-1d059b9679c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:42.691660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278caec5-eebd-4d61-a9db-1d059b9679c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:42.693671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278caec5-eebd-4d61-a9db-1d059b9679c9 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:42.696702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:42.728914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24859d-2bad-4439-a30e-08ea5f33f96f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:42.731679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24859d-2bad-4439-a30e-08ea5f33f96f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:42.790912 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:42.824339 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 87/175] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:17:42.830766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278caec5-eebd-4d61-a9db-1d059b9679c9 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:42.944739 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 89/176] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:17:42.950301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24859d-2bad-4439-a30e-08ea5f33f96f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:43.012801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278caec5-eebd-4d61-a9db-1d059b9679c9 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:43.078056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:43.089394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24859d-2bad-4439-a30e-08ea5f33f96f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:43.092388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e375c0-d354-4446-a1a6-620e9c94a335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:43.093097 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e375c0-d354-4446-a1a6-620e9c94a335 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:43.093413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e375c0-d354-4446-a1a6-620e9c94a335 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:43.093686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e375c0-d354-4446-a1a6-620e9c94a335 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:43.095011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e375c0-d354-4446-a1a6-620e9c94a335 None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:43.096466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e375c0-d354-4446-a1a6-620e9c94a335 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:43.098511 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 88/177] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:42 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:43.103972 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e375c0-d354-4446-a1a6-620e9c94a335 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:43.159082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:43.166802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132aeb20-02c5-45f5-b4f6-75d8396a6354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:43.167459 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132aeb20-02c5-45f5-b4f6-75d8396a6354 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:43.167736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132aeb20-02c5-45f5-b4f6-75d8396a6354 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:43.168002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132aeb20-02c5-45f5-b4f6-75d8396a6354 None admin] PATH_INFO: `/v3/projects/a2abfc5ca6da4956ac25b8040e6b4e58` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:43.176700 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132aeb20-02c5-45f5-b4f6-75d8396a6354 None admin] RBAC: Authorizing `identity:get_project(project_id=a2abfc5ca6da4956ac25b8040e6b4e58)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:43.178166 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132aeb20-02c5-45f5-b4f6-75d8396a6354 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:43.180090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 90/178] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:42 2022] GET /identity/v3/projects/a2abfc5ca6da4956ac25b8040e6b4e58 => generated 312 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:43.187283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132aeb20-02c5-45f5-b4f6-75d8396a6354 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:43.284448 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e375c0-d354-4446-a1a6-620e9c94a335 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:43.374345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c65078-6a24-4417-ba0c-c810e8cfee94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:43.375061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c65078-6a24-4417-ba0c-c810e8cfee94 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:43.375346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c65078-6a24-4417-ba0c-c810e8cfee94 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:43.375616 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c65078-6a24-4417-ba0c-c810e8cfee94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:43.376366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c65078-6a24-4417-ba0c-c810e8cfee94 None admin] RBAC: Adding query filter params (role.id=0d40b3176f78460f9c4a29c1e9ef843e, scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b, user.id=6ca6948ab5ab429ab41c690eaa6cc6c5) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:43.378234 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c65078-6a24-4417-ba0c-c810e8cfee94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:43.383656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c65078-6a24-4417-ba0c-c810e8cfee94 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:43.392274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 89/179] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:43 2022] GET /identity/v3/role_assignments?user.id=6ca6948ab5ab429ab41c690eaa6cc6c5&scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b&role.id=0d40b3176f78460f9c4a29c1e9ef843e => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:43.406542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132aeb20-02c5-45f5-b4f6-75d8396a6354 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:43.517438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd4447-866c-45f3-aaad-52fe616484c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:43.518079 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd4447-866c-45f3-aaad-52fe616484c6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:43.518355 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd4447-866c-45f3-aaad-52fe616484c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:43.518685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd4447-866c-45f3-aaad-52fe616484c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:43.519403 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd4447-866c-45f3-aaad-52fe616484c6 None admin] RBAC: Adding query filter params (role.id=9e02fa627e5c4388b0aa7182f1707895, scope.project.id=a2abfc5ca6da4956ac25b8040e6b4e58, user.id=6ca6948ab5ab429ab41c690eaa6cc6c5) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:43.519708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd4447-866c-45f3-aaad-52fe616484c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:43.521161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd4447-866c-45f3-aaad-52fe616484c6 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:43.541779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 91/180] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:43 2022] GET /identity/v3/role_assignments?user.id=6ca6948ab5ab429ab41c690eaa6cc6c5&scope.project.id=a2abfc5ca6da4956ac25b8040e6b4e58&role.id=9e02fa627e5c4388b0aa7182f1707895 => generated 608 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:44.684516 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d790d-ce1e-4f95-a0ab-bf42184dc404 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:44.685074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d790d-ce1e-4f95-a0ab-bf42184dc404 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:44.685074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d790d-ce1e-4f95-a0ab-bf42184dc404 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:44.686427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 90/181] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:44.692614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dab27-c2c2-4ba9-80bf-911e874486e1 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:44.693511 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dab27-c2c2-4ba9-80bf-911e874486e1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:44.693998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dab27-c2c2-4ba9-80bf-911e874486e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:44.719202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-502dab27-c2c2-4ba9-80bf-911e874486e1 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:44.720417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-502dab27-c2c2-4ba9-80bf-911e874486e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:44.743813 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:44.793341 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 92/182] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:44.835537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa1365-e1ea-4fef-bc11-d0ceb6f15965 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:44.835842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa1365-e1ea-4fef-bc11-d0ceb6f15965 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:44.836087 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa1365-e1ea-4fef-bc11-d0ceb6f15965 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:44.837456 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 91/183] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:44.842695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e80cf-a450-43ff-ac1d-8e6ea77478e3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:44.842981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e80cf-a450-43ff-ac1d-8e6ea77478e3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:44.843249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e80cf-a450-43ff-ac1d-8e6ea77478e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:44.861596 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c13e80cf-a450-43ff-ac1d-8e6ea77478e3 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:44.862814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c13e80cf-a450-43ff-ac1d-8e6ea77478e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:44.936624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 93/184] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:45.070940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa1365-e1ea-4fef-bc11-d0ceb6f15965 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:45.141660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e01bd3-58b3-49e1-aa6d-7fa59c83686a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:45.141991 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e01bd3-58b3-49e1-aa6d-7fa59c83686a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:45.142196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e01bd3-58b3-49e1-aa6d-7fa59c83686a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:45.143769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 94/185] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:45.148561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e088f5-a85a-403f-b9cf-7ad359deb7ec None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:45.148842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e088f5-a85a-403f-b9cf-7ad359deb7ec None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:45.149110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e088f5-a85a-403f-b9cf-7ad359deb7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:45.167794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f9e088f5-a85a-403f-b9cf-7ad359deb7ec None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:45.168967 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f9e088f5-a85a-403f-b9cf-7ad359deb7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:45.174866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:45.178658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa1365-e1ea-4fef-bc11-d0ceb6f15965 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:45.215325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a1fba-8414-400c-be39-303a12c4f40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:45.215895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a1fba-8414-400c-be39-303a12c4f40f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:45.216170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a1fba-8414-400c-be39-303a12c4f40f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:45.216436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a1fba-8414-400c-be39-303a12c4f40f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:45.229377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a1fba-8414-400c-be39-303a12c4f40f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-a76a1fba-8414-400c-be39-303a12c4f40f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:45.234423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:45.235660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:45.236624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:45.236624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:45.236624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:45.236624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:45.236624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:45.236624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:45.236624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:45.236624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:45.236624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:45.236624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:45.236624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:45.236624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:45.236624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 92/186] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:17:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:45.236624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a1fba-8414-400c-be39-303a12c4f40f None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:45.251495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 95/187] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:17:45.255420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e088f5-a85a-403f-b9cf-7ad359deb7ec None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:45.272955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:45.352814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a1fba-8414-400c-be39-303a12c4f40f None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:45.359704 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e088f5-a85a-403f-b9cf-7ad359deb7ec None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:45.387238 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1099224-2afc-4ad2-8f2c-38d58edd480d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:45.387882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1099224-2afc-4ad2-8f2c-38d58edd480d None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:45.388218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1099224-2afc-4ad2-8f2c-38d58edd480d None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:45.388429 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1099224-2afc-4ad2-8f2c-38d58edd480d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:45.389560 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1099224-2afc-4ad2-8f2c-38d58edd480d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:45.391048 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1099224-2afc-4ad2-8f2c-38d58edd480d None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:45.392700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 93/188] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:45.395806 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0ae47-a9aa-4c5c-bd76-efdd03f90b86 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:45.396083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0ae47-a9aa-4c5c-bd76-efdd03f90b86 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:45.396385 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0ae47-a9aa-4c5c-bd76-efdd03f90b86 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:45.397828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 94/189] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:45.398748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb72e9b-fcaf-4c9d-b189-ad3742c52c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:45.399364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb72e9b-fcaf-4c9d-b189-ad3742c52c60 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:45.399641 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb72e9b-fcaf-4c9d-b189-ad3742c52c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:45.399909 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb72e9b-fcaf-4c9d-b189-ad3742c52c60 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:45.400569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb72e9b-fcaf-4c9d-b189-ad3742c52c60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:45.400864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb72e9b-fcaf-4c9d-b189-ad3742c52c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:45.402242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb72e9b-fcaf-4c9d-b189-ad3742c52c60 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:45.402753 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0ae47-a9aa-4c5c-bd76-efdd03f90b86 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:45.415191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 96/190] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:45.415191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ede56c-aefa-43e0-88ac-2879a7ba2a91 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:45.415191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ede56c-aefa-43e0-88ac-2879a7ba2a91 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:45.415191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ede56c-aefa-43e0-88ac-2879a7ba2a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:45.430731 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a2ede56c-aefa-43e0-88ac-2879a7ba2a91 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:45.431411 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:45.431895 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a2ede56c-aefa-43e0-88ac-2879a7ba2a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:45.500358 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 97/191] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:45.500937 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ede56c-aefa-43e0-88ac-2879a7ba2a91 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:45.511047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:45.512652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0ae47-a9aa-4c5c-bd76-efdd03f90b86 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:45.552507 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122577b-e5bf-4905-a065-935502c97a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:45.553149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122577b-e5bf-4905-a065-935502c97a95 None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:45.553421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122577b-e5bf-4905-a065-935502c97a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:45.553706 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122577b-e5bf-4905-a065-935502c97a95 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:45.554702 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122577b-e5bf-4905-a065-935502c97a95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:45.556143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122577b-e5bf-4905-a065-935502c97a95 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:45.578348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 95/192] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:17:45 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:17:45.582809 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122577b-e5bf-4905-a065-935502c97a95 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:45.621983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ede56c-aefa-43e0-88ac-2879a7ba2a91 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:45.660456 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40413e18-84aa-4dfe-90b5-6dc441d09f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:45.661434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40413e18-84aa-4dfe-90b5-6dc441d09f39 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:45.661843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40413e18-84aa-4dfe-90b5-6dc441d09f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:45.662249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40413e18-84aa-4dfe-90b5-6dc441d09f39 None admin] PATH_INFO: `/v3/users/aab8612a10154f62a7dcc414a3548b3c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:45.677679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40413e18-84aa-4dfe-90b5-6dc441d09f39 None admin] RBAC: Authorizing `identity:get_user(user_id=aab8612a10154f62a7dcc414a3548b3c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:45.679583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40413e18-84aa-4dfe-90b5-6dc441d09f39 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:45.681497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 98/193] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:45 2022] GET /identity/v3/users/aab8612a10154f62a7dcc414a3548b3c => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:45.690533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40413e18-84aa-4dfe-90b5-6dc441d09f39 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:45.690713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:45.711566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122577b-e5bf-4905-a065-935502c97a95 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:45.754633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c209095-69ad-4735-9d2c-d216137287b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:45.755529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c209095-69ad-4735-9d2c-d216137287b0 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:45.755933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c209095-69ad-4735-9d2c-d216137287b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:45.756181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c209095-69ad-4735-9d2c-d216137287b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:45.764663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:45.775291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c209095-69ad-4735-9d2c-d216137287b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-8c209095-69ad-4735-9d2c-d216137287b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:45.780787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:45.781652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:45.782412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:45.782412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:45.782412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:45.782412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:45.782412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:45.782412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:45.782412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:45.782412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:45.782412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:45.782412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:45.782412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:17:45.782412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:45.782412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 96/194] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:17:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:45.783218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c209095-69ad-4735-9d2c-d216137287b0 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:45.813937 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40413e18-84aa-4dfe-90b5-6dc441d09f39 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:45.851110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9a4bc-a9e5-43c6-afa3-986ffcc749b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:45.852026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9a4bc-a9e5-43c6-afa3-986ffcc749b5 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:45.852724 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9a4bc-a9e5-43c6-afa3-986ffcc749b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:45.852999 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9a4bc-a9e5-43c6-afa3-986ffcc749b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:45.910862 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c209095-69ad-4735-9d2c-d216137287b0 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:45.946041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc9a4bc-a9e5-43c6-afa3-986ffcc749b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:45.947715 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc9a4bc-a9e5-43c6-afa3-986ffcc749b5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:45.951866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033dae39-0c84-42c5-a215-5f75f923e1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:45.952522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dae39-0c84-42c5-a215-5f75f923e1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:45.952789 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dae39-0c84-42c5-a215-5f75f923e1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:45.953053 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dae39-0c84-42c5-a215-5f75f923e1f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:45.964533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033dae39-0c84-42c5-a215-5f75f923e1f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-033dae39-0c84-42c5-a215-5f75f923e1f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:45.969798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:45.970769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:45.971581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:45.971581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:45.971581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:45.971581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:45.971581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:45.971581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:45.971581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:45.971581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:45.971581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:45.971581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:45.971581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:17:45.971581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:45.971581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 97/195] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:45.972388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033dae39-0c84-42c5-a215-5f75f923e1f9 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:45.974379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:46.037401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:46.079917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033dae39-0c84-42c5-a215-5f75f923e1f9 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:46.096151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 99/196] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:17:46.097497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9a4bc-a9e5-43c6-afa3-986ffcc749b5 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:46.123208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0103f-053c-45ae-903e-f888971c00ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:46.123880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0103f-053c-45ae-903e-f888971c00ab None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:46.124161 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0103f-053c-45ae-903e-f888971c00ab None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:46.124427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0103f-053c-45ae-903e-f888971c00ab None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:46.125139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0103f-053c-45ae-903e-f888971c00ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:46.125451 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0103f-053c-45ae-903e-f888971c00ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:46.126877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0103f-053c-45ae-903e-f888971c00ab None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:46.136050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 98/197] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:17:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:46.136691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0103f-053c-45ae-903e-f888971c00ab None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:46.202929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:46.238287 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9a4bc-a9e5-43c6-afa3-986ffcc749b5 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:46.242249 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0103f-053c-45ae-903e-f888971c00ab None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:46.286548 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d781e7-55b6-4354-bea4-6cd15f0bba0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:46.287359 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d781e7-55b6-4354-bea4-6cd15f0bba0e None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:46.287669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7120568-6270-46c8-b428-48f5aa95b45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:46.287795 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d781e7-55b6-4354-bea4-6cd15f0bba0e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:46.288244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d781e7-55b6-4354-bea4-6cd15f0bba0e None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:46.288734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7120568-6270-46c8-b428-48f5aa95b45c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:46.289267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7120568-6270-46c8-b428-48f5aa95b45c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:46.289671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7120568-6270-46c8-b428-48f5aa95b45c None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:46.289890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d781e7-55b6-4354-bea4-6cd15f0bba0e None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:46.290705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7120568-6270-46c8-b428-48f5aa95b45c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:46.291201 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7120568-6270-46c8-b428-48f5aa95b45c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:46.291652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d781e7-55b6-4354-bea4-6cd15f0bba0e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:46.292868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7120568-6270-46c8-b428-48f5aa95b45c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:46.293492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 99/198] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:46 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:46.298221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d781e7-55b6-4354-bea4-6cd15f0bba0e None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:46.310142 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 100/199] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:46.312031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7120568-6270-46c8-b428-48f5aa95b45c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:46.354438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:46.403954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d781e7-55b6-4354-bea4-6cd15f0bba0e None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:46.434176 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:46.439631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ec026-a0f6-481e-844b-c88adc5eac5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:46.440279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ec026-a0f6-481e-844b-c88adc5eac5e None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:46.440579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ec026-a0f6-481e-844b-c88adc5eac5e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:46.440841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7120568-6270-46c8-b428-48f5aa95b45c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:46.441041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ec026-a0f6-481e-844b-c88adc5eac5e None admin] PATH_INFO: `/v3/users/aab8612a10154f62a7dcc414a3548b3c` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:46.443286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ec026-a0f6-481e-844b-c88adc5eac5e None admin] RBAC: Authorizing `identity:get_user(user_id=aab8612a10154f62a7dcc414a3548b3c)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:46.445451 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ec026-a0f6-481e-844b-c88adc5eac5e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:46.447094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 100/200] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:46 2022] GET /identity/v3/users/aab8612a10154f62a7dcc414a3548b3c => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:46.447912 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ec026-a0f6-481e-844b-c88adc5eac5e None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:46.479697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1dd90-1bbb-434c-b907-3fff75d3cb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:46.480597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1dd90-1bbb-434c-b907-3fff75d3cb09 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:46.481012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1dd90-1bbb-434c-b907-3fff75d3cb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:46.481514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1dd90-1bbb-434c-b907-3fff75d3cb09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:46.482459 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1dd90-1bbb-434c-b907-3fff75d3cb09 None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=0b1e73957f8a494d88eab174721e9401, user.id=aab8612a10154f62a7dcc414a3548b3c) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:46.483018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1dd90-1bbb-434c-b907-3fff75d3cb09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:46.484699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1dd90-1bbb-434c-b907-3fff75d3cb09 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:46.490790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 101/201] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:46 2022] GET /identity/v3/role_assignments?user.id=aab8612a10154f62a7dcc414a3548b3c&scope.project.id=0b1e73957f8a494d88eab174721e9401&role.id=c6da7ba2be504943af29338ee8855f05 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:46.499672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1dd90-1bbb-434c-b907-3fff75d3cb09 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:46.566654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ec026-a0f6-481e-844b-c88adc5eac5e None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:46.596874 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9c177-e83d-45b9-a7fc-54758be92cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:46.597496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9c177-e83d-45b9-a7fc-54758be92cce None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:46.597773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9c177-e83d-45b9-a7fc-54758be92cce None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:46.598041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9c177-e83d-45b9-a7fc-54758be92cce None admin] PATH_INFO: `/v3/users/3b2028839dfb45e8947efd3bdbd9ab5f` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:46.599259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9c177-e83d-45b9-a7fc-54758be92cce None admin] RBAC: Authorizing `identity:get_user(user_id=3b2028839dfb45e8947efd3bdbd9ab5f)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:46.600757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9c177-e83d-45b9-a7fc-54758be92cce None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:46.602930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 101/202] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:46 2022] GET /identity/v3/users/3b2028839dfb45e8947efd3bdbd9ab5f => generated 289 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:46.609348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9c177-e83d-45b9-a7fc-54758be92cce None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:46.709632 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:46.728034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:46.728635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1dd90-1bbb-434c-b907-3fff75d3cb09 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:46.764242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9c177-e83d-45b9-a7fc-54758be92cce None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:46.782934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbbae9-e147-4b8f-8b5b-697203afbb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:46.783574 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbbae9-e147-4b8f-8b5b-697203afbb9a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:46.783826 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbbae9-e147-4b8f-8b5b-697203afbb9a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:46.784096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbbae9-e147-4b8f-8b5b-697203afbb9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:46.804044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece88d1-d12c-4885-a614-dfd683d6dba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:46.804667 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece88d1-d12c-4885-a614-dfd683d6dba6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:46.804941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece88d1-d12c-4885-a614-dfd683d6dba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:46.805206 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece88d1-d12c-4885-a614-dfd683d6dba6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:46.896985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece88d1-d12c-4885-a614-dfd683d6dba6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:46.898546 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece88d1-d12c-4885-a614-dfd683d6dba6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:46.904397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edbbae9-e147-4b8f-8b5b-697203afbb9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:46.914523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edbbae9-e147-4b8f-8b5b-697203afbb9a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:46.970525 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:47.002549 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 102/203] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 16:17:47.005555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece88d1-d12c-4885-a614-dfd683d6dba6 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:47.114705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:47.132701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 102/204] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:17:47.138871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbbae9-e147-4b8f-8b5b-697203afbb9a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:47.173006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece88d1-d12c-4885-a614-dfd683d6dba6 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:47.242305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176572d-5dca-42e0-bfe9-665e9504fb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:47.243670 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176572d-5dca-42e0-bfe9-665e9504fb44 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:47.243944 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176572d-5dca-42e0-bfe9-665e9504fb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:47.244210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176572d-5dca-42e0-bfe9-665e9504fb44 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:47.246092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176572d-5dca-42e0-bfe9-665e9504fb44 None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:47.247575 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176572d-5dca-42e0-bfe9-665e9504fb44 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:47.249215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 103/205] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:47 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:47.257143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176572d-5dca-42e0-bfe9-665e9504fb44 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:47.327448 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:47.337021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbbae9-e147-4b8f-8b5b-697203afbb9a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:47.407316 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69325f07-520e-46b3-a86a-844775473508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:47.407981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69325f07-520e-46b3-a86a-844775473508 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:47.408260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69325f07-520e-46b3-a86a-844775473508 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:47.408526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69325f07-520e-46b3-a86a-844775473508 None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:47.412335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69325f07-520e-46b3-a86a-844775473508 None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:47.413833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69325f07-520e-46b3-a86a-844775473508 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:47.419097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 103/206] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:47 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:47.424895 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69325f07-520e-46b3-a86a-844775473508 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:47.440841 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176572d-5dca-42e0-bfe9-665e9504fb44 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:47.477638 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f0493-b21b-475b-a227-314a4719a26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:47.478301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f0493-b21b-475b-a227-314a4719a26f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:47.478608 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f0493-b21b-475b-a227-314a4719a26f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:47.478879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f0493-b21b-475b-a227-314a4719a26f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:47.479627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f0493-b21b-475b-a227-314a4719a26f None admin] RBAC: Adding query filter params (role.id=17a32575480045789f8f279a3ada4fd1, scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b, user.id=3b2028839dfb45e8947efd3bdbd9ab5f) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:47.479914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f0493-b21b-475b-a227-314a4719a26f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:47.481474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f0493-b21b-475b-a227-314a4719a26f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:47.494760 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 104/207] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:47 2022] GET /identity/v3/role_assignments?user.id=3b2028839dfb45e8947efd3bdbd9ab5f&scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b&role.id=17a32575480045789f8f279a3ada4fd1 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:47.527470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:47.572226 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69325f07-520e-46b3-a86a-844775473508 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:47.608300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c837329-d2d7-4543-a701-702abd6717d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:47.609319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c837329-d2d7-4543-a701-702abd6717d1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:47.609799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c837329-d2d7-4543-a701-702abd6717d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:47.610280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c837329-d2d7-4543-a701-702abd6717d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:47.611376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c837329-d2d7-4543-a701-702abd6717d1 None admin] RBAC: Adding query filter params (role.id=0d40b3176f78460f9c4a29c1e9ef843e, scope.project.id=0b1e73957f8a494d88eab174721e9401, user.id=aab8612a10154f62a7dcc414a3548b3c) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:47.611916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c837329-d2d7-4543-a701-702abd6717d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:47.613627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c837329-d2d7-4543-a701-702abd6717d1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:47.620323 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 104/208] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:47 2022] GET /identity/v3/role_assignments?user.id=aab8612a10154f62a7dcc414a3548b3c&scope.project.id=0b1e73957f8a494d88eab174721e9401&role.id=0d40b3176f78460f9c4a29c1e9ef843e => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:48.159325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91281c93-e566-4cc0-b959-d750884887fd None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:48.159838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91281c93-e566-4cc0-b959-d750884887fd None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:48.159911 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91281c93-e566-4cc0-b959-d750884887fd None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:48.161473 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 105/209] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:48.167264 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db57f6-337f-48a2-b2e7-d2b24bd346ca None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:48.167723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db57f6-337f-48a2-b2e7-d2b24bd346ca None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:48.167798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db57f6-337f-48a2-b2e7-d2b24bd346ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:48.182875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b358e8-29a7-456d-a005-a8c94de75eb1 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:48.183183 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b358e8-29a7-456d-a005-a8c94de75eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:48.183423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b358e8-29a7-456d-a005-a8c94de75eb1 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:48.185035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 106/210] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:48.190210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae1c4b-f031-41fe-a950-c004fc9552a3 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:48.190478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae1c4b-f031-41fe-a950-c004fc9552a3 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:48.190831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae1c4b-f031-41fe-a950-c004fc9552a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:48.191191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-50db57f6-337f-48a2-b2e7-d2b24bd346ca None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:48.192738 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-50db57f6-337f-48a2-b2e7-d2b24bd346ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:48.209843 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-22ae1c4b-f031-41fe-a950-c004fc9552a3 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:48.211126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-22ae1c4b-f031-41fe-a950-c004fc9552a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:48.282797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 105/211] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:48.288787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 107/212] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:17:48.551011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db57f6-337f-48a2-b2e7-d2b24bd346ca None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:48.563140 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae1c4b-f031-41fe-a950-c004fc9552a3 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:48.604060 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:48.609565 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:48.664050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db57f6-337f-48a2-b2e7-d2b24bd346ca None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:48.700551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff135f-4451-42cc-b4d7-cbe14f61fd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:48.701692 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff135f-4451-42cc-b4d7-cbe14f61fd5e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:48.702183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff135f-4451-42cc-b4d7-cbe14f61fd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:48.702683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff135f-4451-42cc-b4d7-cbe14f61fd5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:48.703864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae1c4b-f031-41fe-a950-c004fc9552a3 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:48.716318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff135f-4451-42cc-b4d7-cbe14f61fd5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-cbff135f-4451-42cc-b4d7-cbe14f61fd5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:48.722271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:48.723225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:48.724000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:48.724000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:48.724000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:48.724000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:48.724000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:48.724000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:48.724000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:48.724000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:48.724000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:48.724000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:48.724000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:48.724000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:48.731759 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 106/213] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:48.733216 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff135f-4451-42cc-b4d7-cbe14f61fd5e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:48.746794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c7429-4ecc-44a2-b0c6-e2fe6532cfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:48.747451 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c7429-4ecc-44a2-b0c6-e2fe6532cfe8 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:48.747757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c7429-4ecc-44a2-b0c6-e2fe6532cfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:48.748412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c7429-4ecc-44a2-b0c6-e2fe6532cfe8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:48.749816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c7429-4ecc-44a2-b0c6-e2fe6532cfe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:48.751578 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c7429-4ecc-44a2-b0c6-e2fe6532cfe8 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:48.753970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 108/214] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:48.760024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c7429-4ecc-44a2-b0c6-e2fe6532cfe8 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:48.850151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff135f-4451-42cc-b4d7-cbe14f61fd5e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:48.863999 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:48.880016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:48.883839 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c7429-4ecc-44a2-b0c6-e2fe6532cfe8 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:48.885874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fd40c-3d5d-40ff-a8ca-ee256e21147e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:48.886491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fd40c-3d5d-40ff-a8ca-ee256e21147e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:48.886793 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fd40c-3d5d-40ff-a8ca-ee256e21147e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:48.887115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fd40c-3d5d-40ff-a8ca-ee256e21147e None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:48.887785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fd40c-3d5d-40ff-a8ca-ee256e21147e None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:48.888084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fd40c-3d5d-40ff-a8ca-ee256e21147e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:48.889597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fd40c-3d5d-40ff-a8ca-ee256e21147e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:48.897568 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 107/215] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:48.898819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5143dbd-7619-47e8-adec-5ac3120a46b9 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:48.899098 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5143dbd-7619-47e8-adec-5ac3120a46b9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:48.899380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5143dbd-7619-47e8-adec-5ac3120a46b9 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:48.900548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 108/216] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:48.906685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5143dbd-7619-47e8-adec-5ac3120a46b9 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:48.930152 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c2fa8-c332-41ee-b874-22a1c6eb0173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:48.931073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c2fa8-c332-41ee-b874-22a1c6eb0173 None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:48.931598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c2fa8-c332-41ee-b874-22a1c6eb0173 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:48.932011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c2fa8-c332-41ee-b874-22a1c6eb0173 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:48.933053 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c2fa8-c332-41ee-b874-22a1c6eb0173 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:48.935392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c2fa8-c332-41ee-b874-22a1c6eb0173 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:48.968070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 109/217] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:17:48 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:17:48.968070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13b9b0-60e9-4c52-9a39-46efda930047 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:48.968070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13b9b0-60e9-4c52-9a39-46efda930047 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:48.968070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13b9b0-60e9-4c52-9a39-46efda930047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:48.986077 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-ff13b9b0-60e9-4c52-9a39-46efda930047 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:48.987333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ff13b9b0-60e9-4c52-9a39-46efda930047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:49.021282 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5143dbd-7619-47e8-adec-5ac3120a46b9 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:49.063140 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbae8d5-2502-4999-aad6-9ee946fb7aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:49.063932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbae8d5-2502-4999-aad6-9ee946fb7aff None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:49.064268 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbae8d5-2502-4999-aad6-9ee946fb7aff None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:49.064602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbae8d5-2502-4999-aad6-9ee946fb7aff None admin] PATH_INFO: `/v3/users/9096913cef3540668b6f6df022b9abe7` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:49.084420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 110/218] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:49.094024 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbae8d5-2502-4999-aad6-9ee946fb7aff None admin] RBAC: Authorizing `identity:get_user(user_id=9096913cef3540668b6f6df022b9abe7)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:49.095950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbae8d5-2502-4999-aad6-9ee946fb7aff None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:49.103160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 109/219] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:48 2022] GET /identity/v3/users/9096913cef3540668b6f6df022b9abe7 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:49.104459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13b9b0-60e9-4c52-9a39-46efda930047 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:49.196211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:49.212861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13b9b0-60e9-4c52-9a39-46efda930047 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:49.249202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a510c3c-ea1f-47ee-809a-5cb752ebbff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:49.249824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a510c3c-ea1f-47ee-809a-5cb752ebbff4 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:49.250104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a510c3c-ea1f-47ee-809a-5cb752ebbff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:49.250378 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a510c3c-ea1f-47ee-809a-5cb752ebbff4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:49.327233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a510c3c-ea1f-47ee-809a-5cb752ebbff4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:49.329046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a510c3c-ea1f-47ee-809a-5cb752ebbff4 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:49.367382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbae8d5-2502-4999-aad6-9ee946fb7aff None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:49.415233 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:49.436206 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:49.459371 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 111/220] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:17:49.465168 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a510c3c-ea1f-47ee-809a-5cb752ebbff4 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:49.525625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbae8d5-2502-4999-aad6-9ee946fb7aff None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:49.606780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb82f0-0571-4d2c-88b8-0d2755201a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:49.607473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb82f0-0571-4d2c-88b8-0d2755201a96 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:49.607745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb82f0-0571-4d2c-88b8-0d2755201a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:49.608052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb82f0-0571-4d2c-88b8-0d2755201a96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:49.614927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a510c3c-ea1f-47ee-809a-5cb752ebbff4 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:49.633701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb82f0-0571-4d2c-88b8-0d2755201a96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-7ecb82f0-0571-4d2c-88b8-0d2755201a96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:49.638567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:49.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:49.640599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:49.640599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:49.640599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:49.640599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:49.640599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:49.640599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:49.640599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:49.640599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:49.640599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:49.640599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:49.640599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:49.640599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:49.640599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 110/221] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:17:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:17:49.648004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb82f0-0571-4d2c-88b8-0d2755201a96 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:49.669069 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4a900-9f6e-40ac-940a-9221a63144db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:49.669718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4a900-9f6e-40ac-940a-9221a63144db None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:49.669992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4a900-9f6e-40ac-940a-9221a63144db None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:49.670261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4a900-9f6e-40ac-940a-9221a63144db None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:49.672248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4a900-9f6e-40ac-940a-9221a63144db None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:49.673711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4a900-9f6e-40ac-940a-9221a63144db None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:49.677119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 112/222] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:49 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:49.683539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4a900-9f6e-40ac-940a-9221a63144db None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:49.771837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:49.828960 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:49.838784 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb82f0-0571-4d2c-88b8-0d2755201a96 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:49.854595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4a900-9f6e-40ac-940a-9221a63144db None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:49.903716 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe8a51-8ede-4263-ad99-1869b3354875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:49.904266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe8a51-8ede-4263-ad99-1869b3354875 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:49.904538 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe8a51-8ede-4263-ad99-1869b3354875 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:49.904807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe8a51-8ede-4263-ad99-1869b3354875 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:49.905510 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe8a51-8ede-4263-ad99-1869b3354875 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:49.905806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe8a51-8ede-4263-ad99-1869b3354875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:49.907295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe8a51-8ede-4263-ad99-1869b3354875 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:49.920770 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5c960-9804-4d9c-8122-de66f9167df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:49.921392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5c960-9804-4d9c-8122-de66f9167df1 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:49.921663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5c960-9804-4d9c-8122-de66f9167df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:49.921926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5c960-9804-4d9c-8122-de66f9167df1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:49.922682 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5c960-9804-4d9c-8122-de66f9167df1 None admin] RBAC: Adding query filter params (role.id=9e02fa627e5c4388b0aa7182f1707895, scope.project.id=0b1e73957f8a494d88eab174721e9401, user.id=9096913cef3540668b6f6df022b9abe7) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:49.922982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5c960-9804-4d9c-8122-de66f9167df1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:49.924428 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5c960-9804-4d9c-8122-de66f9167df1 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:49.930632 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 111/223] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:49.934199 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 113/224] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:49 2022] GET /identity/v3/role_assignments?user.id=9096913cef3540668b6f6df022b9abe7&scope.project.id=0b1e73957f8a494d88eab174721e9401&role.id=9e02fa627e5c4388b0aa7182f1707895 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:49.945227 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe8a51-8ede-4263-ad99-1869b3354875 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:50.084973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df236d9c-81a8-4e2e-9240-4e9cfd2ef0e8 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:50.085334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df236d9c-81a8-4e2e-9240-4e9cfd2ef0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:50.085606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df236d9c-81a8-4e2e-9240-4e9cfd2ef0e8 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:50.087027 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 114/225] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:50.093263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfacd8-3904-4b3f-9f91-9a15cd010391 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:50.093739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfacd8-3904-4b3f-9f91-9a15cd010391 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:50.094084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfacd8-3904-4b3f-9f91-9a15cd010391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:50.098992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe8a51-8ede-4263-ad99-1869b3354875 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:50.116268 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-27bfacd8-3904-4b3f-9f91-9a15cd010391 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:50.118908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-27bfacd8-3904-4b3f-9f91-9a15cd010391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:50.168117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d8569-b965-4576-916c-a1b562f15cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:50.169255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d8569-b965-4576-916c-a1b562f15cfb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:50.169763 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d8569-b965-4576-916c-a1b562f15cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:50.170409 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d8569-b965-4576-916c-a1b562f15cfb None admin] PATH_INFO: `/v3/users/aab8612a10154f62a7dcc414a3548b3c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:50.172481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d8569-b965-4576-916c-a1b562f15cfb None admin] RBAC: Authorizing `identity:get_user(user_id=aab8612a10154f62a7dcc414a3548b3c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:50.174528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d8569-b965-4576-916c-a1b562f15cfb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:50.176903 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 112/226] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:49 2022] GET /identity/v3/users/aab8612a10154f62a7dcc414a3548b3c => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:50.182806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d8569-b965-4576-916c-a1b562f15cfb None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:50.222518 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:50.241174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 115/227] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:17:50.309080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d8569-b965-4576-916c-a1b562f15cfb None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:50.352886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14710f46-16c1-4fa2-8990-2b4191fe7ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:50.353774 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14710f46-16c1-4fa2-8990-2b4191fe7ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:50.354199 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14710f46-16c1-4fa2-8990-2b4191fe7ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:50.354632 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14710f46-16c1-4fa2-8990-2b4191fe7ad9 None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:50.359156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14710f46-16c1-4fa2-8990-2b4191fe7ad9 None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:50.360782 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14710f46-16c1-4fa2-8990-2b4191fe7ad9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:50.363175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 113/228] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:50 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:50.369226 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfacd8-3904-4b3f-9f91-9a15cd010391 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:50.448069 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:50.509730 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfacd8-3904-4b3f-9f91-9a15cd010391 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:50.543386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8e22b-9d1e-47c8-894c-8350bf826b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:50.544057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8e22b-9d1e-47c8-894c-8350bf826b6a None admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:50.544334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8e22b-9d1e-47c8-894c-8350bf826b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:50.544602 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8e22b-9d1e-47c8-894c-8350bf826b6a None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401/users/aab8612a10154f62a7dcc414a3548b3c/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:50.546537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8e22b-9d1e-47c8-894c-8350bf826b6a None admin] RBAC: Authorizing `identity:create_grant(project_id=0b1e73957f8a494d88eab174721e9401, user_id=aab8612a10154f62a7dcc414a3548b3c, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:50.548015 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8e22b-9d1e-47c8-894c-8350bf826b6a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:50.573896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 116/229] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:17:50 2022] PUT /identity/v3/projects/0b1e73957f8a494d88eab174721e9401/users/aab8612a10154f62a7dcc414a3548b3c/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:17:50.613364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14710f46-16c1-4fa2-8990-2b4191fe7ad9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:50.759627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:50.771408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14710f46-16c1-4fa2-8990-2b4191fe7ad9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:50.811605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cad29-9744-4465-9dd3-75af7093b694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:50.812250 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cad29-9744-4465-9dd3-75af7093b694 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:50.812523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cad29-9744-4465-9dd3-75af7093b694 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:50.812792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cad29-9744-4465-9dd3-75af7093b694 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:50.827776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15cad29-9744-4465-9dd3-75af7093b694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-f15cad29-9744-4465-9dd3-75af7093b694 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:50.833071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:50.833965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:50.834792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:50.834792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:50.834792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:50.834792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:50.834792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:50.834792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:50.834792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:50.834792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:50.834792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:50.834792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:50.834792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:17:50.834792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:50.834792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 114/230] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:17:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:50.841229 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8e22b-9d1e-47c8-894c-8350bf826b6a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:50.955788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8e22b-9d1e-47c8-894c-8350bf826b6a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:50.967659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:51.001155 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc82789-fe38-4ef4-8d7d-140168cb5ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:51.001774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc82789-fe38-4ef4-8d7d-140168cb5ada None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:51.002047 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc82789-fe38-4ef4-8d7d-140168cb5ada None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:51.002313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc82789-fe38-4ef4-8d7d-140168cb5ada None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:51.003007 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc82789-fe38-4ef4-8d7d-140168cb5ada None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:51.003304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc82789-fe38-4ef4-8d7d-140168cb5ada None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:51.004750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc82789-fe38-4ef4-8d7d-140168cb5ada None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:51.017215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 117/231] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:17:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:51.022931 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cad29-9744-4465-9dd3-75af7093b694 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:51.163961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cad29-9744-4465-9dd3-75af7093b694 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:51.214334 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd1ea8-f686-442d-b518-780274caa123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:51.215054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd1ea8-f686-442d-b518-780274caa123 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:51.215330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd1ea8-f686-442d-b518-780274caa123 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:51.215595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd1ea8-f686-442d-b518-780274caa123 None admin] PATH_INFO: `/v3/users/aab8612a10154f62a7dcc414a3548b3c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:51.216717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd1ea8-f686-442d-b518-780274caa123 None admin] RBAC: Authorizing `identity:get_user(user_id=aab8612a10154f62a7dcc414a3548b3c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:51.218183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd1ea8-f686-442d-b518-780274caa123 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:51.219855 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 115/232] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:51 2022] GET /identity/v3/users/aab8612a10154f62a7dcc414a3548b3c => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:51.226074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc82789-fe38-4ef4-8d7d-140168cb5ada None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:51.339433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc82789-fe38-4ef4-8d7d-140168cb5ada None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:51.385449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0382b6e-7e2a-40a0-ac3f-d8561f7e6af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:51.386110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0382b6e-7e2a-40a0-ac3f-d8561f7e6af4 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:51.386391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0382b6e-7e2a-40a0-ac3f-d8561f7e6af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:51.386690 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0382b6e-7e2a-40a0-ac3f-d8561f7e6af4 None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:51.387895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0382b6e-7e2a-40a0-ac3f-d8561f7e6af4 None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:51.389356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0382b6e-7e2a-40a0-ac3f-d8561f7e6af4 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:51.391242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 118/233] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:51 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:51.397025 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd1ea8-f686-442d-b518-780274caa123 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:51.440751 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:51.626147 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd1ea8-f686-442d-b518-780274caa123 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:51.689950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44570df-239a-4ca1-9882-56270730909c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:51.690677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44570df-239a-4ca1-9882-56270730909c None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:51.690957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44570df-239a-4ca1-9882-56270730909c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:51.691230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44570df-239a-4ca1-9882-56270730909c None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401/users/aab8612a10154f62a7dcc414a3548b3c/roles/0d40b3176f78460f9c4a29c1e9ef843e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:51.699258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44570df-239a-4ca1-9882-56270730909c None admin] RBAC: Authorizing `identity:create_grant(project_id=0b1e73957f8a494d88eab174721e9401, user_id=aab8612a10154f62a7dcc414a3548b3c, role_id=0d40b3176f78460f9c4a29c1e9ef843e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:51.700725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44570df-239a-4ca1-9882-56270730909c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:51.726764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 116/234] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:17:51 2022] PUT /identity/v3/projects/0b1e73957f8a494d88eab174721e9401/users/aab8612a10154f62a7dcc414a3548b3c/roles/0d40b3176f78460f9c4a29c1e9ef843e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:17:51.730419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf827d6-53f6-4770-8689-02854cfd8c64 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:51.730728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf827d6-53f6-4770-8689-02854cfd8c64 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:51.730999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf827d6-53f6-4770-8689-02854cfd8c64 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:51.732711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 119/235] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:51.738269 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26877bb-d981-46fa-9f8c-edc55c39a397 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:51.738647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26877bb-d981-46fa-9f8c-edc55c39a397 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:51.738850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26877bb-d981-46fa-9f8c-edc55c39a397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:51.767773 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b26877bb-d981-46fa-9f8c-edc55c39a397 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:51.768991 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b26877bb-d981-46fa-9f8c-edc55c39a397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:51.876485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 117/236] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:52.082362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acfda0-1c7e-419f-8f77-93578a5441a9 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:52.083057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acfda0-1c7e-419f-8f77-93578a5441a9 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:52.084186 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acfda0-1c7e-419f-8f77-93578a5441a9 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:52.086090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 120/237] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:52.091912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4e6e4-830d-4424-891a-332932a33bc3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:52.092197 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4e6e4-830d-4424-891a-332932a33bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:52.092466 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4e6e4-830d-4424-891a-332932a33bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:52.112664 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-09e4e6e4-830d-4424-891a-332932a33bc3 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:52.113845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-09e4e6e4-830d-4424-891a-332932a33bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:52.118420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:52.209376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acfda0-1c7e-419f-8f77-93578a5441a9 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:52.217009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 118/238] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:52.255992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:52.326711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acfda0-1c7e-419f-8f77-93578a5441a9 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:52.363432 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d6539-3aee-4754-bd6e-6a8331dd13ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:52.364100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d6539-3aee-4754-bd6e-6a8331dd13ca None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:52.364361 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d6539-3aee-4754-bd6e-6a8331dd13ca None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:52.364633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d6539-3aee-4754-bd6e-6a8331dd13ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:52.365773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d6539-3aee-4754-bd6e-6a8331dd13ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:52.367423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d6539-3aee-4754-bd6e-6a8331dd13ca None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:52.369235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 121/239] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:52.376860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4e6e4-830d-4424-891a-332932a33bc3 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:52.501976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45715e92-ab83-4604-b50d-c480db940b83 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:52.502144 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45715e92-ab83-4604-b50d-c480db940b83 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:52.502419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45715e92-ab83-4604-b50d-c480db940b83 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:52.503346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4e6e4-830d-4424-891a-332932a33bc3 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:52.504052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 122/240] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:17:52.508923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb945c-07b1-4e39-bac6-06629fac1104 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:52.509308 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb945c-07b1-4e39-bac6-06629fac1104 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:52.509717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb945c-07b1-4e39-bac6-06629fac1104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:52.531950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-3dbb945c-07b1-4e39-bac6-06629fac1104 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:52.533145 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-3dbb945c-07b1-4e39-bac6-06629fac1104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:52.560118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12de448-953a-49a2-b383-21e1c8fa8212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:52.560736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de448-953a-49a2-b383-21e1c8fa8212 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:52.561010 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de448-953a-49a2-b383-21e1c8fa8212 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:52.561277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de448-953a-49a2-b383-21e1c8fa8212 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:52.561927 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12de448-953a-49a2-b383-21e1c8fa8212 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:52.564559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12de448-953a-49a2-b383-21e1c8fa8212 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:52.590295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:52.618552 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 119/241] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:17:52 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:17:52.623523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12de448-953a-49a2-b383-21e1c8fa8212 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:52.658319 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 123/242] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:52.799138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12de448-953a-49a2-b383-21e1c8fa8212 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:52.835666 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545de0af-9b4e-4bd5-b4ca-7599c2f3a7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:52.836309 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545de0af-9b4e-4bd5-b4ca-7599c2f3a7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:52.836583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545de0af-9b4e-4bd5-b4ca-7599c2f3a7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:52.836846 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545de0af-9b4e-4bd5-b4ca-7599c2f3a7c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:52.848009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545de0af-9b4e-4bd5-b4ca-7599c2f3a7c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-545de0af-9b4e-4bd5-b4ca-7599c2f3a7c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:52.853508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:52.854438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:52.855261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:52.855261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:52.855261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:52.855261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:52.855261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:52.855261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:52.855261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:52.855261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:52.855261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:52.855261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:52.855261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:17:52.855261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:52.855261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 120/243] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:52.860760 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb945c-07b1-4e39-bac6-06629fac1104 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:52.925416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:53.005924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb945c-07b1-4e39-bac6-06629fac1104 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:53.046994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcbe35-2431-467f-ae29-c07cac67eeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:53.047611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcbe35-2431-467f-ae29-c07cac67eeb2 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:53.047969 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcbe35-2431-467f-ae29-c07cac67eeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:53.048238 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcbe35-2431-467f-ae29-c07cac67eeb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:53.048905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcbe35-2431-467f-ae29-c07cac67eeb2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:53.049260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcbe35-2431-467f-ae29-c07cac67eeb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:53.050609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcbe35-2431-467f-ae29-c07cac67eeb2 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:53.060224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 124/244] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:53.066678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545de0af-9b4e-4bd5-b4ca-7599c2f3a7c2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:53.067155 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcbe35-2431-467f-ae29-c07cac67eeb2 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:53.148009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:53.194329 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:53.204994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcbe35-2431-467f-ae29-c07cac67eeb2 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:53.236042 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545de0af-9b4e-4bd5-b4ca-7599c2f3a7c2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:53.252872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e5a9e-7ddf-44e5-8736-3921bb75f9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:53.253522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e5a9e-7ddf-44e5-8736-3921bb75f9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:53.253801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e5a9e-7ddf-44e5-8736-3921bb75f9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:53.254074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e5a9e-7ddf-44e5-8736-3921bb75f9f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:53.268370 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e5a9e-7ddf-44e5-8736-3921bb75f9f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-d00e5a9e-7ddf-44e5-8736-3921bb75f9f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:53.278279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:53.279222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:53.279979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:53.279979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:53.279979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:53.279979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:53.279979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:53.279979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:53.279979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:53.279979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:53.279979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:53.279979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:53.279979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:53.279979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:53.279979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 125/245] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:17:53.284730 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e5a9e-7ddf-44e5-8736-3921bb75f9f5 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:53.286534 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c06018-fbf4-413e-b29d-e7e52be93094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:53.287414 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c06018-fbf4-413e-b29d-e7e52be93094 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:53.287878 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c06018-fbf4-413e-b29d-e7e52be93094 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:53.288389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c06018-fbf4-413e-b29d-e7e52be93094 None admin] PATH_INFO: `/v3/users/52ad7f760538428abb568322ca4266da` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:53.309382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c06018-fbf4-413e-b29d-e7e52be93094 None admin] RBAC: Authorizing `identity:get_user(user_id=52ad7f760538428abb568322ca4266da)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:53.310899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c06018-fbf4-413e-b29d-e7e52be93094 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:53.312864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 121/246] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:53 2022] GET /identity/v3/users/52ad7f760538428abb568322ca4266da => generated 297 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:53.317850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e45410-7d5c-4ffb-b09d-e57defefad4a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:53.318128 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e45410-7d5c-4ffb-b09d-e57defefad4a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:53.318401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e45410-7d5c-4ffb-b09d-e57defefad4a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:53.319844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 122/247] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:53.322021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e45410-7d5c-4ffb-b09d-e57defefad4a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:53.428156 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e5a9e-7ddf-44e5-8736-3921bb75f9f5 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:53.449741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e45410-7d5c-4ffb-b09d-e57defefad4a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:53.465294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:53.475024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd524f4-ae15-469f-9e7c-888eba70b3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:53.476016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd524f4-ae15-469f-9e7c-888eba70b3db None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:53.476466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd524f4-ae15-469f-9e7c-888eba70b3db None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:53.476910 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd524f4-ae15-469f-9e7c-888eba70b3db None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:53.477581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd524f4-ae15-469f-9e7c-888eba70b3db None admin] RBAC: Adding query filter params (name=member) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:53.477884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd524f4-ae15-469f-9e7c-888eba70b3db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:53.479492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd524f4-ae15-469f-9e7c-888eba70b3db None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:53.491381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 126/248] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:53.492978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b83b88-0e50-4dd6-8d18-f7d45a406f7d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:53.493263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b83b88-0e50-4dd6-8d18-f7d45a406f7d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:53.493535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b83b88-0e50-4dd6-8d18-f7d45a406f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:53.493963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb7d9a-b4fe-4fdb-bea5-f084b490d178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:53.494871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb7d9a-b4fe-4fdb-bea5-f084b490d178 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:53.495342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb7d9a-b4fe-4fdb-bea5-f084b490d178 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:53.495799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb7d9a-b4fe-4fdb-bea5-f084b490d178 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:53.527272 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-a3b83b88-0e50-4dd6-8d18-f7d45a406f7d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:53.529654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a3b83b88-0e50-4dd6-8d18-f7d45a406f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:53.558690 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:53.623978 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb7d9a-b4fe-4fdb-bea5-f084b490d178 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:53.625456 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb7d9a-b4fe-4fdb-bea5-f084b490d178 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:53.661034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 127/249] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:53.661906 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b83b88-0e50-4dd6-8d18-f7d45a406f7d None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:53.792701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 123/250] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 16:17:53.800468 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb7d9a-b4fe-4fdb-bea5-f084b490d178 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:53.824918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:53.835603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b83b88-0e50-4dd6-8d18-f7d45a406f7d None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:53.902912 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112e786-50e4-4611-ad55-39332536d23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:53.903474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112e786-50e4-4611-ad55-39332536d23d None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:53.903752 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112e786-50e4-4611-ad55-39332536d23d None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:53.904021 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112e786-50e4-4611-ad55-39332536d23d None admin] PATH_INFO: `/v3/users/9096913cef3540668b6f6df022b9abe7` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:53.905268 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112e786-50e4-4611-ad55-39332536d23d None admin] RBAC: Authorizing `identity:get_user(user_id=9096913cef3540668b6f6df022b9abe7)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:53.908135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112e786-50e4-4611-ad55-39332536d23d None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:53.911301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 128/251] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:53 2022] GET /identity/v3/users/9096913cef3540668b6f6df022b9abe7 => generated 297 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:53.927578 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112e786-50e4-4611-ad55-39332536d23d None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:53.954929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:53.976214 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb7d9a-b4fe-4fdb-bea5-f084b490d178 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:54.033084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49742f-c812-47c7-b4cc-b715ae16c76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:54.033823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49742f-c812-47c7-b4cc-b715ae16c76f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:54.034104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49742f-c812-47c7-b4cc-b715ae16c76f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:54.034375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49742f-c812-47c7-b4cc-b715ae16c76f None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:54.035622 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49742f-c812-47c7-b4cc-b715ae16c76f None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:54.037178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49742f-c812-47c7-b4cc-b715ae16c76f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:54.038905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 124/252] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:53 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:54.039852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49742f-c812-47c7-b4cc-b715ae16c76f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:54.070938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112e786-50e4-4611-ad55-39332536d23d None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:54.111119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816306f-08a5-4b96-a042-20ecb46e54b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:54.111771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816306f-08a5-4b96-a042-20ecb46e54b0 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:54.112048 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816306f-08a5-4b96-a042-20ecb46e54b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:54.112317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816306f-08a5-4b96-a042-20ecb46e54b0 None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:54.114458 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816306f-08a5-4b96-a042-20ecb46e54b0 None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:54.115944 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816306f-08a5-4b96-a042-20ecb46e54b0 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:54.117709 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 129/253] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:53 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:54.118485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816306f-08a5-4b96-a042-20ecb46e54b0 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:54.206314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:54.246437 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49742f-c812-47c7-b4cc-b715ae16c76f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:54.284382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:54.302379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816306f-08a5-4b96-a042-20ecb46e54b0 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:54.321975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c54e34-c2d1-4342-aff8-f9fff258ea5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:54.323118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c54e34-c2d1-4342-aff8-f9fff258ea5b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:54.324970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c54e34-c2d1-4342-aff8-f9fff258ea5b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:54.325380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c54e34-c2d1-4342-aff8-f9fff258ea5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:54.345541 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c54e34-c2d1-4342-aff8-f9fff258ea5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-25c54e34-c2d1-4342-aff8-f9fff258ea5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:54.351644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:54.352701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:54.353629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:54.353629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:54.353629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:54.353629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:54.353629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:54.353629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:54.353629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:54.353629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:54.353629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:54.353629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:54.353629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:17:54.353629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:54.354221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6020a4-10fb-4efb-996d-75ccc9520ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:54.354979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6020a4-10fb-4efb-996d-75ccc9520ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:54.355385 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6020a4-10fb-4efb-996d-75ccc9520ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:54.355765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6020a4-10fb-4efb-996d-75ccc9520ac8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:54.356599 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6020a4-10fb-4efb-996d-75ccc9520ac8 None admin] RBAC: Adding query filter params (role.id=17a32575480045789f8f279a3ada4fd1, scope.project.id=0b1e73957f8a494d88eab174721e9401, user.id=52ad7f760538428abb568322ca4266da) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:54.357018 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6020a4-10fb-4efb-996d-75ccc9520ac8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:54.357707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 125/254] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:17:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:54.358608 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c54e34-c2d1-4342-aff8-f9fff258ea5b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:54.358893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6020a4-10fb-4efb-996d-75ccc9520ac8 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:54.364933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 130/255] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:54 2022] GET /identity/v3/role_assignments?user.id=52ad7f760538428abb568322ca4266da&scope.project.id=0b1e73957f8a494d88eab174721e9401&role.id=17a32575480045789f8f279a3ada4fd1 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:54.365874 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6020a4-10fb-4efb-996d-75ccc9520ac8 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:54.482356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6020a4-10fb-4efb-996d-75ccc9520ac8 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:54.521354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb2462-1596-4528-bbfa-236bc481af20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:54.522234 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb2462-1596-4528-bbfa-236bc481af20 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:54.522732 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb2462-1596-4528-bbfa-236bc481af20 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:54.523137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb2462-1596-4528-bbfa-236bc481af20 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:54.523959 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb2462-1596-4528-bbfa-236bc481af20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:54.524425 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb2462-1596-4528-bbfa-236bc481af20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:54.526001 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb2462-1596-4528-bbfa-236bc481af20 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:54.536404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 131/256] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:54.543620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb2462-1596-4528-bbfa-236bc481af20 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:54.558345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:54.565373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c54e34-c2d1-4342-aff8-f9fff258ea5b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:54.630935 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253fd98-6267-4028-b892-afd6bf3339c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:54.631625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253fd98-6267-4028-b892-afd6bf3339c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:54.631899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253fd98-6267-4028-b892-afd6bf3339c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:54.632221 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253fd98-6267-4028-b892-afd6bf3339c3 None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401/users/9096913cef3540668b6f6df022b9abe7/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:54.635740 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253fd98-6267-4028-b892-afd6bf3339c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b1e73957f8a494d88eab174721e9401, user_id=9096913cef3540668b6f6df022b9abe7, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:54.637253 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253fd98-6267-4028-b892-afd6bf3339c3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:54.663119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 126/257] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:17:54 2022] PUT /identity/v3/projects/0b1e73957f8a494d88eab174721e9401/users/9096913cef3540668b6f6df022b9abe7/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:17:54.664327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d2f5c-67f5-4b93-bef7-af495e21cbb9 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:54.664656 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d2f5c-67f5-4b93-bef7-af495e21cbb9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:54.664931 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d2f5c-67f5-4b93-bef7-af495e21cbb9 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:54.666319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 127/258] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:54.674827 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfad04-a675-4891-acd9-cbf846c75823 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:54.675409 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfad04-a675-4891-acd9-cbf846c75823 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:54.675864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfad04-a675-4891-acd9-cbf846c75823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:54.693176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:54.694796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb2462-1596-4528-bbfa-236bc481af20 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:54.717187 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e5cfad04-a675-4891-acd9-cbf846c75823 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:54.718871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e5cfad04-a675-4891-acd9-cbf846c75823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:54.789968 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015caf5-907d-4e58-8188-c5ebabec52f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:54.790911 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015caf5-907d-4e58-8188-c5ebabec52f7 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:54.791322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015caf5-907d-4e58-8188-c5ebabec52f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:54.791713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015caf5-907d-4e58-8188-c5ebabec52f7 None admin] PATH_INFO: `/v3/users/aab8612a10154f62a7dcc414a3548b3c` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:54.792987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015caf5-907d-4e58-8188-c5ebabec52f7 None admin] RBAC: Authorizing `identity:get_user(user_id=aab8612a10154f62a7dcc414a3548b3c)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:54.794609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015caf5-907d-4e58-8188-c5ebabec52f7 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:54.796693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 132/259] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:54 2022] GET /identity/v3/users/aab8612a10154f62a7dcc414a3548b3c => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:54.808600 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015caf5-907d-4e58-8188-c5ebabec52f7 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:54.819373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 128/260] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:54.936850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:54.961718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015caf5-907d-4e58-8188-c5ebabec52f7 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:54.995397 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce6e2d-5f72-4ddd-9e50-5754cc53a8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:54.996009 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce6e2d-5f72-4ddd-9e50-5754cc53a8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:54.996286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce6e2d-5f72-4ddd-9e50-5754cc53a8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:54.996555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce6e2d-5f72-4ddd-9e50-5754cc53a8b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:55.065572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce6e2d-5f72-4ddd-9e50-5754cc53a8b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:55.067357 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce6e2d-5f72-4ddd-9e50-5754cc53a8b7 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:55.166307 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:55.190137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 133/261] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 16:17:55.197365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfad04-a675-4891-acd9-cbf846c75823 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:55.314929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfad04-a675-4891-acd9-cbf846c75823 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:55.349684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:55.354938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c17d51-0946-4fb7-992b-69408513fd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:55.355580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c17d51-0946-4fb7-992b-69408513fd74 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:55.356003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c17d51-0946-4fb7-992b-69408513fd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:55.356282 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c17d51-0946-4fb7-992b-69408513fd74 None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:55.361189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce6e2d-5f72-4ddd-9e50-5754cc53a8b7 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:55.361561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c17d51-0946-4fb7-992b-69408513fd74 None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:55.363068 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c17d51-0946-4fb7-992b-69408513fd74 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:55.365036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 129/262] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:55 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:55.372352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c17d51-0946-4fb7-992b-69408513fd74 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:55.548595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c17d51-0946-4fb7-992b-69408513fd74 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:55.554324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce6e2d-5f72-4ddd-9e50-5754cc53a8b7 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:55.594936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151a341-7642-4f41-9214-458a20aceb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:55.595900 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151a341-7642-4f41-9214-458a20aceb43 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:55.596311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151a341-7642-4f41-9214-458a20aceb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:55.596708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151a341-7642-4f41-9214-458a20aceb43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:55.607446 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151a341-7642-4f41-9214-458a20aceb43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-5151a341-7642-4f41-9214-458a20aceb43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:55.612685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:55.613584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:55.614362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:55.614362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:55.614362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:55.614362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:55.614362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:55.614362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:55.614362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:55.614362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:55.614362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:55.614362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:55.614362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:17:55.614362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:55.617129 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 134/263] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:17:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:55.623497 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151a341-7642-4f41-9214-458a20aceb43 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:55.651229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530199f-e307-414f-b338-12ff0a6e2263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:55.651867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530199f-e307-414f-b338-12ff0a6e2263 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:55.652145 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530199f-e307-414f-b338-12ff0a6e2263 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:55.652414 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530199f-e307-414f-b338-12ff0a6e2263 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:55.653164 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530199f-e307-414f-b338-12ff0a6e2263 None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=0b1e73957f8a494d88eab174721e9401, user.id=aab8612a10154f62a7dcc414a3548b3c) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:55.653464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530199f-e307-414f-b338-12ff0a6e2263 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:55.654974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530199f-e307-414f-b338-12ff0a6e2263 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:55.673760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 130/264] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:55 2022] GET /identity/v3/role_assignments?user.id=aab8612a10154f62a7dcc414a3548b3c&scope.project.id=0b1e73957f8a494d88eab174721e9401&role.id=c6da7ba2be504943af29338ee8855f05 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:55.694558 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:55.785103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151a341-7642-4f41-9214-458a20aceb43 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:55.831151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b265683-5dea-42a6-ab47-b510dee5ad96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:55.832106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b265683-5dea-42a6-ab47-b510dee5ad96 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:55.832555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b265683-5dea-42a6-ab47-b510dee5ad96 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:55.833076 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b265683-5dea-42a6-ab47-b510dee5ad96 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:55.833922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b265683-5dea-42a6-ab47-b510dee5ad96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:55.834412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b265683-5dea-42a6-ab47-b510dee5ad96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:55.836081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b265683-5dea-42a6-ab47-b510dee5ad96 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:55.845891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 135/265] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:17:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:55.851730 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530199f-e307-414f-b338-12ff0a6e2263 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:55.944003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:55.988896 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530199f-e307-414f-b338-12ff0a6e2263 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:56.031690 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205cce3-c015-4b20-aee0-3da24f3c7dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:56.032351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205cce3-c015-4b20-aee0-3da24f3c7dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:56.032627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205cce3-c015-4b20-aee0-3da24f3c7dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:56.032898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205cce3-c015-4b20-aee0-3da24f3c7dd8 None admin] PATH_INFO: `/v3/users/aab8612a10154f62a7dcc414a3548b3c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:56.034101 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205cce3-c015-4b20-aee0-3da24f3c7dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=aab8612a10154f62a7dcc414a3548b3c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:56.035658 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205cce3-c015-4b20-aee0-3da24f3c7dd8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:56.037735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 131/266] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:55 2022] GET /identity/v3/users/aab8612a10154f62a7dcc414a3548b3c => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:56.043169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b265683-5dea-42a6-ab47-b510dee5ad96 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:56.117383 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379652f4-f9f2-440d-9275-701d6cc25c40 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:56.117705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379652f4-f9f2-440d-9275-701d6cc25c40 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:56.117945 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379652f4-f9f2-440d-9275-701d6cc25c40 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:56.119478 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 132/267] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:17:56.124722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755503da-d87f-4c60-a7f5-20658d1e6481 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:56.125000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755503da-d87f-4c60-a7f5-20658d1e6481 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:56.125275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755503da-d87f-4c60-a7f5-20658d1e6481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:56.148415 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-755503da-d87f-4c60-a7f5-20658d1e6481 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:56.149670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-755503da-d87f-4c60-a7f5-20658d1e6481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:56.172691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:56.175855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b265683-5dea-42a6-ab47-b510dee5ad96 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:56.217360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ed57f-48bd-4c4b-9776-b156a3fd4f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:56.217898 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ed57f-48bd-4c4b-9776-b156a3fd4f2a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:56.218171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ed57f-48bd-4c4b-9776-b156a3fd4f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:56.218433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ed57f-48bd-4c4b-9776-b156a3fd4f2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:56.244892 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 133/268] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:17:56.247251 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f12cf-143e-4662-817d-c4b4cc9cffd4 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:56.247537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f12cf-143e-4662-817d-c4b4cc9cffd4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:56.248085 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f12cf-143e-4662-817d-c4b4cc9cffd4 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:56.249810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 134/269] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:56.255886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dcc71-a187-4ddc-a557-0cc31d4fced8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:56.256175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dcc71-a187-4ddc-a557-0cc31d4fced8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:56.256434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dcc71-a187-4ddc-a557-0cc31d4fced8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:56.278721 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ac0dcc71-a187-4ddc-a557-0cc31d4fced8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:56.279906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ac0dcc71-a187-4ddc-a557-0cc31d4fced8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:56.293188 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ed57f-48bd-4c4b-9776-b156a3fd4f2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:56.294753 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ed57f-48bd-4c4b-9776-b156a3fd4f2a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:56.382893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:56.399124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 135/270] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:17:56.441071 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 136/271] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:17:56.441374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:56.447257 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0dcc71-a187-4ddc-a557-0cc31d4fced8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:56.566233 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0dcc71-a187-4ddc-a557-0cc31d4fced8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:56.579624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ed57f-48bd-4c4b-9776-b156a3fd4f2a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:56.612353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a35345-02a4-4752-b853-cb4f860e3a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:56.613003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a35345-02a4-4752-b853-cb4f860e3a6f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:56.613281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a35345-02a4-4752-b853-cb4f860e3a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:56.613550 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a35345-02a4-4752-b853-cb4f860e3a6f None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:56.615326 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a35345-02a4-4752-b853-cb4f860e3a6f None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:56.616841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a35345-02a4-4752-b853-cb4f860e3a6f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:56.619207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 136/272] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:56 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:56.625177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a35345-02a4-4752-b853-cb4f860e3a6f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:56.709054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:56.716488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ed57f-48bd-4c4b-9776-b156a3fd4f2a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:56.749526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a35345-02a4-4752-b853-cb4f860e3a6f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:56.753306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e0b74-afec-4e72-9c0b-b413f484566e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:56.754187 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e0b74-afec-4e72-9c0b-b413f484566e None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:56.754701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e0b74-afec-4e72-9c0b-b413f484566e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:56.755154 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e0b74-afec-4e72-9c0b-b413f484566e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:56.756445 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e0b74-afec-4e72-9c0b-b413f484566e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:56.758234 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e0b74-afec-4e72-9c0b-b413f484566e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:56.760334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 137/273] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:17:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:56.761276 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e0b74-afec-4e72-9c0b-b413f484566e None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:56.787801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec013143-326d-4d50-a363-fd19d1549f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:56.788804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec013143-326d-4d50-a363-fd19d1549f7c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:56.789276 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec013143-326d-4d50-a363-fd19d1549f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:56.789736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec013143-326d-4d50-a363-fd19d1549f7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:56.790712 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec013143-326d-4d50-a363-fd19d1549f7c None admin] RBAC: Adding query filter params (role.id=0d40b3176f78460f9c4a29c1e9ef843e, scope.project.id=0b1e73957f8a494d88eab174721e9401, user.id=aab8612a10154f62a7dcc414a3548b3c) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:56.791212 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec013143-326d-4d50-a363-fd19d1549f7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:56.792866 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec013143-326d-4d50-a363-fd19d1549f7c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:56.806674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 137/274] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:56 2022] GET /identity/v3/role_assignments?user.id=aab8612a10154f62a7dcc414a3548b3c&scope.project.id=0b1e73957f8a494d88eab174721e9401&role.id=0d40b3176f78460f9c4a29c1e9ef843e => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:56.807976 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec013143-326d-4d50-a363-fd19d1549f7c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:56.843003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:56.871116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e0b74-afec-4e72-9c0b-b413f484566e None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:56.909540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25774ecd-5796-4434-a626-aa6541d9838b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:56.910431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25774ecd-5796-4434-a626-aa6541d9838b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:56.910898 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25774ecd-5796-4434-a626-aa6541d9838b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:56.911301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25774ecd-5796-4434-a626-aa6541d9838b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:56.923942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25774ecd-5796-4434-a626-aa6541d9838b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-25774ecd-5796-4434-a626-aa6541d9838b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:56.929877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:56.930859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:56.931628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:56.931628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:56.931628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:56.931628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:56.931628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:56.931628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:56.931628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:56.931628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:56.931628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:56.931628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:56.931628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:56.931628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:56.934484 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 138/275] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:56.940024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25774ecd-5796-4434-a626-aa6541d9838b None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:57.054026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25774ecd-5796-4434-a626-aa6541d9838b None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:57.093948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:57.101439 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36685e-93f1-4f63-bc07-02ab4c96b1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:57.102749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36685e-93f1-4f63-bc07-02ab4c96b1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:57.103695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36685e-93f1-4f63-bc07-02ab4c96b1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:57.104102 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36685e-93f1-4f63-bc07-02ab4c96b1b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:57.105234 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36685e-93f1-4f63-bc07-02ab4c96b1b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:57.105898 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36685e-93f1-4f63-bc07-02ab4c96b1b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:57.108105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36685e-93f1-4f63-bc07-02ab4c96b1b9 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:57.109669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec013143-326d-4d50-a363-fd19d1549f7c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:57.120015 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 139/276] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:57.126120 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36685e-93f1-4f63-bc07-02ab4c96b1b9 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:57.139268 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:57.153170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d669e-9794-42d3-9690-d66dafec15f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:57.153947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d669e-9794-42d3-9690-d66dafec15f5 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:57.154319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d669e-9794-42d3-9690-d66dafec15f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:57.154725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d669e-9794-42d3-9690-d66dafec15f5 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:57.155499 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d669e-9794-42d3-9690-d66dafec15f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:57.157047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d669e-9794-42d3-9690-d66dafec15f5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:57.184435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 138/277] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:17:56 2022] POST /identity/v3/users => generated 293 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:17:57.252533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36685e-93f1-4f63-bc07-02ab4c96b1b9 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:57.296170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07f951-03e1-41de-9940-266f831bda08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:57.296855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07f951-03e1-41de-9940-266f831bda08 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:57.297139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07f951-03e1-41de-9940-266f831bda08 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:57.297458 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07f951-03e1-41de-9940-266f831bda08 None admin] PATH_INFO: `/v3/users/30c9e9bf80b341df88707678dda9947c` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:57.314027 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07f951-03e1-41de-9940-266f831bda08 None admin] RBAC: Authorizing `identity:get_user(user_id=30c9e9bf80b341df88707678dda9947c)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:57.315617 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07f951-03e1-41de-9940-266f831bda08 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:57.322632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 140/278] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:57 2022] GET /identity/v3/users/30c9e9bf80b341df88707678dda9947c => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:57.324385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d669e-9794-42d3-9690-d66dafec15f5 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:57.439604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d669e-9794-42d3-9690-d66dafec15f5 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:57.479218 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b63fa4-3398-459e-a634-7aa6ee3f1ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:57.480032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b63fa4-3398-459e-a634-7aa6ee3f1ced None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:57.480321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b63fa4-3398-459e-a634-7aa6ee3f1ced None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:57.480600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b63fa4-3398-459e-a634-7aa6ee3f1ced None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:57.481353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b63fa4-3398-459e-a634-7aa6ee3f1ced None admin] RBAC: Adding query filter params (role.id=9e02fa627e5c4388b0aa7182f1707895, user.id=30c9e9bf80b341df88707678dda9947c, scope.system=all) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:57.481670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b63fa4-3398-459e-a634-7aa6ee3f1ced None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:57.483276 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b63fa4-3398-459e-a634-7aa6ee3f1ced None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:57.489580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 139/279] 10.222.0.23 () {56 vars in 1308 bytes} [Fri Sep 9 16:17:57 2022] GET /identity/v3/role_assignments?user.id=30c9e9bf80b341df88707678dda9947c&scope.system=all&role.id=9e02fa627e5c4388b0aa7182f1707895 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:57.578030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc652afa-85ec-4348-99c6-7eab977f9c3a None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:57.578321 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc652afa-85ec-4348-99c6-7eab977f9c3a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:57.578628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc652afa-85ec-4348-99c6-7eab977f9c3a None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:57.580338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 141/280] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:57.587211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f65e5c-d3f2-4956-88c2-82d5c3c227fa None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:57.587535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f65e5c-d3f2-4956-88c2-82d5c3c227fa None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:57.587764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f65e5c-d3f2-4956-88c2-82d5c3c227fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:57.610732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-52f65e5c-d3f2-4956-88c2-82d5c3c227fa None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:57.611996 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-52f65e5c-d3f2-4956-88c2-82d5c3c227fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:57.664662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:57.695128 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 140/281] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:57.725092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa960845-a547-4450-a316-f306ac5cb136 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:57.725375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa960845-a547-4450-a316-f306ac5cb136 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:57.725677 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa960845-a547-4450-a316-f306ac5cb136 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:57.727320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 142/282] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:57.732687 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d957f-6cb2-4ad6-8cb0-9a3524aad708 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:57.732969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d957f-6cb2-4ad6-8cb0-9a3524aad708 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:57.733242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d957f-6cb2-4ad6-8cb0-9a3524aad708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:57.752362 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e57d957f-6cb2-4ad6-8cb0-9a3524aad708 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:57.753553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e57d957f-6cb2-4ad6-8cb0-9a3524aad708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:57.839840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 141/283] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:17:57.965315 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa960845-a547-4450-a316-f306ac5cb136 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:58.020458 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:58.086730 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa960845-a547-4450-a316-f306ac5cb136 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:58.140874 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1d8ad-dc11-4b2b-992f-839e6c641b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:58.141873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1d8ad-dc11-4b2b-992f-839e6c641b23 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:58.142353 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1d8ad-dc11-4b2b-992f-839e6c641b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:58.142864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1d8ad-dc11-4b2b-992f-839e6c641b23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:58.149464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d957f-6cb2-4ad6-8cb0-9a3524aad708 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:58.162414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1d8ad-dc11-4b2b-992f-839e6c641b23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-84f1d8ad-dc11-4b2b-992f-839e6c641b23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:58.167765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:58.168645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:58.169414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:58.169414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:58.169414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:58.169414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:58.169414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:58.169414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:58.169414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:58.169414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:58.169414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:58.169414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:58.169414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:17:58.169414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:17:58.171783 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 143/284] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:58.179521 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1d8ad-dc11-4b2b-992f-839e6c641b23 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:58.252441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d957f-6cb2-4ad6-8cb0-9a3524aad708 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:58.291592 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9187778-8d09-46e7-963f-93707e0de686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:58.292290 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9187778-8d09-46e7-963f-93707e0de686 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:58.292567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9187778-8d09-46e7-963f-93707e0de686 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:58.292880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9187778-8d09-46e7-963f-93707e0de686 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:58.298905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:58.307639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9187778-8d09-46e7-963f-93707e0de686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c9187778-8d09-46e7-963f-93707e0de686 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:17:58.312873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:17:58.313748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:17:58.314561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:17:58.314561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:17:58.314561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:17:58.314561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:17:58.314561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:17:58.314561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:17:58.314561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:17:58.314561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:17:58.314561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:17:58.314561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:17:58.314561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:17:58.314561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:17:58.314561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 142/285] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:17:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:17:58.321514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9187778-8d09-46e7-963f-93707e0de686 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:58.382963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1d8ad-dc11-4b2b-992f-839e6c641b23 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:58.393528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:58.420646 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114090c-ab3a-446d-8c3f-849ea32e4e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:58.421271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114090c-ab3a-446d-8c3f-849ea32e4e1c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:58.421548 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114090c-ab3a-446d-8c3f-849ea32e4e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:58.421815 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114090c-ab3a-446d-8c3f-849ea32e4e1c None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:58.423018 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114090c-ab3a-446d-8c3f-849ea32e4e1c None admin] RBAC: Adding query filter params (name=member) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:58.423333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114090c-ab3a-446d-8c3f-849ea32e4e1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:58.430533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114090c-ab3a-446d-8c3f-849ea32e4e1c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:58.435767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 144/286] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:58.443018 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114090c-ab3a-446d-8c3f-849ea32e4e1c None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:58.545394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9187778-8d09-46e7-963f-93707e0de686 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:58.595266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d60bd-2f5c-4b51-b91f-6d4a7bcf9047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:58.595877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d60bd-2f5c-4b51-b91f-6d4a7bcf9047 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:58.596397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d60bd-2f5c-4b51-b91f-6d4a7bcf9047 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:58.596666 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d60bd-2f5c-4b51-b91f-6d4a7bcf9047 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:58.598207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d60bd-2f5c-4b51-b91f-6d4a7bcf9047 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:58.598540 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d60bd-2f5c-4b51-b91f-6d4a7bcf9047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:58.599945 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d60bd-2f5c-4b51-b91f-6d4a7bcf9047 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:58.609554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 143/287] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:17:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:58.615130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d60bd-2f5c-4b51-b91f-6d4a7bcf9047 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:58.647437 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114090c-ab3a-446d-8c3f-849ea32e4e1c None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:58.661849 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:58.712201 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143df93d-4f29-4c3c-bef5-5e3061f4eeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:58.712844 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143df93d-4f29-4c3c-bef5-5e3061f4eeb4 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:58.713119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143df93d-4f29-4c3c-bef5-5e3061f4eeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:58.713387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143df93d-4f29-4c3c-bef5-5e3061f4eeb4 None admin] PATH_INFO: `/v3/users/9096913cef3540668b6f6df022b9abe7` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:58.715811 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143df93d-4f29-4c3c-bef5-5e3061f4eeb4 None admin] RBAC: Authorizing `identity:get_user(user_id=9096913cef3540668b6f6df022b9abe7)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:58.717256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143df93d-4f29-4c3c-bef5-5e3061f4eeb4 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:58.718895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 145/288] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:58 2022] GET /identity/v3/users/9096913cef3540668b6f6df022b9abe7 => generated 297 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:58.724461 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143df93d-4f29-4c3c-bef5-5e3061f4eeb4 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:58.769069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d60bd-2f5c-4b51-b91f-6d4a7bcf9047 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:58.778773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:58.826066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001661ce-b213-4c10-a15e-b6cf30ab0e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:58.827186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001661ce-b213-4c10-a15e-b6cf30ab0e4d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:58.827720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001661ce-b213-4c10-a15e-b6cf30ab0e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:58.828183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001661ce-b213-4c10-a15e-b6cf30ab0e4d None admin] PATH_INFO: `/v3/users/52ad7f760538428abb568322ca4266da` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:58.829826 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001661ce-b213-4c10-a15e-b6cf30ab0e4d None admin] RBAC: Authorizing `identity:get_user(user_id=52ad7f760538428abb568322ca4266da)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:58.831519 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001661ce-b213-4c10-a15e-b6cf30ab0e4d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:58.840666 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 144/289] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:17:58 2022] GET /identity/v3/users/52ad7f760538428abb568322ca4266da => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:58.846244 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001661ce-b213-4c10-a15e-b6cf30ab0e4d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:58.892310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143df93d-4f29-4c3c-bef5-5e3061f4eeb4 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:58.957853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40992a48-586e-45ac-a13e-b780e1925ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:58.959089 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40992a48-586e-45ac-a13e-b780e1925ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:58.960334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40992a48-586e-45ac-a13e-b780e1925ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:58.961416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40992a48-586e-45ac-a13e-b780e1925ee9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:59.033259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:59.045114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001661ce-b213-4c10-a15e-b6cf30ab0e4d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:59.090520 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40992a48-586e-45ac-a13e-b780e1925ee9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:59.091218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40992a48-586e-45ac-a13e-b780e1925ee9 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:59.114557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249126f-93f6-46cc-a69f-0857912fb7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:59.115206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249126f-93f6-46cc-a69f-0857912fb7aa None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:59.115485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249126f-93f6-46cc-a69f-0857912fb7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:59.115754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249126f-93f6-46cc-a69f-0857912fb7aa None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:59.117687 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249126f-93f6-46cc-a69f-0857912fb7aa None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:59.119185 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249126f-93f6-46cc-a69f-0857912fb7aa None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:59.120834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 145/290] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:58 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:59.126734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249126f-93f6-46cc-a69f-0857912fb7aa None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:59.166894 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:59.237170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 146/291] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:17:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:17:59.242810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40992a48-586e-45ac-a13e-b780e1925ee9 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:59.340872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40992a48-586e-45ac-a13e-b780e1925ee9 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:59.347596 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249126f-93f6-46cc-a69f-0857912fb7aa None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:59.379186 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2a1f2-3606-4ad3-a3be-f514f2ec8235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:59.379833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2a1f2-3606-4ad3-a3be-f514f2ec8235 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:59.380111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2a1f2-3606-4ad3-a3be-f514f2ec8235 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:59.380389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2a1f2-3606-4ad3-a3be-f514f2ec8235 None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:59.383272 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2a1f2-3606-4ad3-a3be-f514f2ec8235 None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:59.384817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2a1f2-3606-4ad3-a3be-f514f2ec8235 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:59.386757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 147/292] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:17:59 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:17:59.397258 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2a1f2-3606-4ad3-a3be-f514f2ec8235 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:17:59.414843 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:59.426133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadae4e5-a55b-4dc6-a283-3e82fc7981fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:59.426834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadae4e5-a55b-4dc6-a283-3e82fc7981fb None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:59.427114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadae4e5-a55b-4dc6-a283-3e82fc7981fb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:59.427390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadae4e5-a55b-4dc6-a283-3e82fc7981fb None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401/users/52ad7f760538428abb568322ca4266da/roles/17a32575480045789f8f279a3ada4fd1` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:59.431575 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadae4e5-a55b-4dc6-a283-3e82fc7981fb None admin] RBAC: Authorizing `identity:create_grant(project_id=0b1e73957f8a494d88eab174721e9401, user_id=52ad7f760538428abb568322ca4266da, role_id=17a32575480045789f8f279a3ada4fd1)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:59.433053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadae4e5-a55b-4dc6-a283-3e82fc7981fb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:59.456958 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 146/293] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:17:59 2022] PUT /identity/v3/projects/0b1e73957f8a494d88eab174721e9401/users/52ad7f760538428abb568322ca4266da/roles/17a32575480045789f8f279a3ada4fd1 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:17:59.533084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2a1f2-3606-4ad3-a3be-f514f2ec8235 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:17:59.571462 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e413e19-41df-4ede-a0d0-bfade13efa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:17:59.572090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e413e19-41df-4ede-a0d0-bfade13efa6b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:59.572370 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e413e19-41df-4ede-a0d0-bfade13efa6b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:59.572642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e413e19-41df-4ede-a0d0-bfade13efa6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:59.573437 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e413e19-41df-4ede-a0d0-bfade13efa6b None admin] RBAC: Adding query filter params (role.id=9e02fa627e5c4388b0aa7182f1707895, scope.project.id=0b1e73957f8a494d88eab174721e9401, user.id=9096913cef3540668b6f6df022b9abe7) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:17:59.573747 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e413e19-41df-4ede-a0d0-bfade13efa6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:17:59.575262 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e413e19-41df-4ede-a0d0-bfade13efa6b None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:17:59.582251 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 148/294] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:17:59 2022] GET /identity/v3/role_assignments?user.id=9096913cef3540668b6f6df022b9abe7&scope.project.id=0b1e73957f8a494d88eab174721e9401&role.id=9e02fa627e5c4388b0aa7182f1707895 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:17:59.879762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c233a71-c22a-4175-9f4f-9cf8b6c3ee4a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:59.880137 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c233a71-c22a-4175-9f4f-9cf8b6c3ee4a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:59.880381 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c233a71-c22a-4175-9f4f-9cf8b6c3ee4a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:59.882194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 147/295] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:59.887640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c1ec8-f2ed-45a4-a2e1-281983a4e9ac None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:59.888224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c1ec8-f2ed-45a4-a2e1-281983a4e9ac None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:59.888224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c1ec8-f2ed-45a4-a2e1-281983a4e9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:59.911250 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-9c9c1ec8-f2ed-45a4-a2e1-281983a4e9ac None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:17:59.912440 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-9c9c1ec8-f2ed-45a4-a2e1-281983a4e9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:17:59.979394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56349107-7f83-4eb2-8de5-2002bf88e05e None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:59.979732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56349107-7f83-4eb2-8de5-2002bf88e05e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:59.979944 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56349107-7f83-4eb2-8de5-2002bf88e05e None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:59.981559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 148/296] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:17:59.986725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b6081-5de3-406d-9103-2a58f55e6524 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:17:59.987249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b6081-5de3-406d-9103-2a58f55e6524 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:17:59.987711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b6081-5de3-406d-9103-2a58f55e6524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:17:59.992778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:17:59.998489 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 149/297] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:00.014069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1b0b6081-5de3-406d-9103-2a58f55e6524 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:00.015699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1b0b6081-5de3-406d-9103-2a58f55e6524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:00.033991 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:00.102116 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 149/298] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:18:00.287046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b58ac-4fe3-46f6-ac2d-f6f00d859491 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:00.287369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b58ac-4fe3-46f6-ac2d-f6f00d859491 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:00.287537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b58ac-4fe3-46f6-ac2d-f6f00d859491 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:00.289222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 150/299] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:00.294623 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33038dd9-6746-48e1-8762-ceba60a80af4 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:00.294910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33038dd9-6746-48e1-8762-ceba60a80af4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:00.295183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33038dd9-6746-48e1-8762-ceba60a80af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:00.295866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b58ac-4fe3-46f6-ac2d-f6f00d859491 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:00.312955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-33038dd9-6746-48e1-8762-ceba60a80af4 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:00.314131 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-33038dd9-6746-48e1-8762-ceba60a80af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:00.394914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 150/300] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:00.396595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33038dd9-6746-48e1-8762-ceba60a80af4 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:00.412340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b58ac-4fe3-46f6-ac2d-f6f00d859491 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:00.448741 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d01ecd-6e36-4dae-a4de-1240a76776f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:00.449443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d01ecd-6e36-4dae-a4de-1240a76776f7 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:00.449721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d01ecd-6e36-4dae-a4de-1240a76776f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:00.449991 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d01ecd-6e36-4dae-a4de-1240a76776f7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:00.460887 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d01ecd-6e36-4dae-a4de-1240a76776f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-84d01ecd-6e36-4dae-a4de-1240a76776f7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:00.465904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:00.466813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:00.472983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:00.472983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:00.472983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:00.472983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:00.472983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:00.472983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:00.472983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:00.472983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:00.472983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:00.472983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:00.472983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:18:00.472983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:00.472983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 151/301] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:00.472983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d01ecd-6e36-4dae-a4de-1240a76776f7 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:00.519705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:00.542148 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33038dd9-6746-48e1-8762-ceba60a80af4 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:00.550574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:00.580203 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67679c4-247c-48f8-bf7f-021bb0c9dd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:00.581032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67679c4-247c-48f8-bf7f-021bb0c9dd73 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:00.581374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67679c4-247c-48f8-bf7f-021bb0c9dd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:00.581708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67679c4-247c-48f8-bf7f-021bb0c9dd73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:00.583003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67679c4-247c-48f8-bf7f-021bb0c9dd73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:00.584520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67679c4-247c-48f8-bf7f-021bb0c9dd73 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:00.586847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 151/302] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:00.592811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67679c4-247c-48f8-bf7f-021bb0c9dd73 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:00.602948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d01ecd-6e36-4dae-a4de-1240a76776f7 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:00.636934 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aa100-2512-43ee-abe8-08337909d651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:00.637550 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aa100-2512-43ee-abe8-08337909d651 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:00.637826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aa100-2512-43ee-abe8-08337909d651 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:00.638095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aa100-2512-43ee-abe8-08337909d651 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:00.638798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38aa100-2512-43ee-abe8-08337909d651 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:00.639102 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38aa100-2512-43ee-abe8-08337909d651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:00.640541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38aa100-2512-43ee-abe8-08337909d651 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:00.649093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 152/303] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:18:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:00.654493 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aa100-2512-43ee-abe8-08337909d651 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:00.714572 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67679c4-247c-48f8-bf7f-021bb0c9dd73 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:00.746215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bc7d3-2d13-4b87-911c-c291bacdaa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:00.746866 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bc7d3-2d13-4b87-911c-c291bacdaa0c None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:00.747146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bc7d3-2d13-4b87-911c-c291bacdaa0c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:00.747417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bc7d3-2d13-4b87-911c-c291bacdaa0c None admin] PATH_INFO: `/v3/groups` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:00.748038 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4bc7d3-2d13-4b87-911c-c291bacdaa0c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:00.749474 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4bc7d3-2d13-4b87-911c-c291bacdaa0c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:00.762918 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aa100-2512-43ee-abe8-08337909d651 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:00.770670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 152/304] 10.222.0.23 () {60 vars in 1134 bytes} [Fri Sep 9 16:18:00 2022] POST /identity/v3/groups => generated 229 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:18:00.771651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bc7d3-2d13-4b87-911c-c291bacdaa0c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:00.786183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:00.801228 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:00.815045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11851063-98b7-4d18-bddf-7b2b064b1e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:00.816110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11851063-98b7-4d18-bddf-7b2b064b1e81 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:00.816564 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11851063-98b7-4d18-bddf-7b2b064b1e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:00.817380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11851063-98b7-4d18-bddf-7b2b064b1e81 None admin] PATH_INFO: `/v3/users/b8543bdc118f405d8bed154f2b5af95a` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:00.832769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11851063-98b7-4d18-bddf-7b2b064b1e81 None admin] RBAC: Authorizing `identity:get_user(user_id=b8543bdc118f405d8bed154f2b5af95a)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:00.834636 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11851063-98b7-4d18-bddf-7b2b064b1e81 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:00.837179 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 153/305] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:18:00 2022] GET /identity/v3/users/b8543bdc118f405d8bed154f2b5af95a => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:00.843305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11851063-98b7-4d18-bddf-7b2b064b1e81 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:00.910816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bc7d3-2d13-4b87-911c-c291bacdaa0c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:00.963435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943582c-5073-45ae-b0df-cb4129233174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:00.964972 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943582c-5073-45ae-b0df-cb4129233174 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:00.965255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943582c-5073-45ae-b0df-cb4129233174 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:00.967871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943582c-5073-45ae-b0df-cb4129233174 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:00.971762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11851063-98b7-4d18-bddf-7b2b064b1e81 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:00.980976 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943582c-5073-45ae-b0df-cb4129233174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-e943582c-5073-45ae-b0df-cb4129233174 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:00.987545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:00.988408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:00.989190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:00.989190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:00.989190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:00.989190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:00.989190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:00.989190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:00.989190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:00.989190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:00.989190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:00.989190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:00.989190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:00.989190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:00.989190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 153/306] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:00.995622 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943582c-5073-45ae-b0df-cb4129233174 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:01.010015 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89607a21-6920-4d7c-a20a-6fc0fab92a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:01.010730 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89607a21-6920-4d7c-a20a-6fc0fab92a1f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:01.011071 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89607a21-6920-4d7c-a20a-6fc0fab92a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:01.011413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89607a21-6920-4d7c-a20a-6fc0fab92a1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:01.012207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89607a21-6920-4d7c-a20a-6fc0fab92a1f None admin] RBAC: Adding query filter params (role.id=17a32575480045789f8f279a3ada4fd1, user.id=b8543bdc118f405d8bed154f2b5af95a, scope.system=all) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:01.012509 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89607a21-6920-4d7c-a20a-6fc0fab92a1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:01.013947 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89607a21-6920-4d7c-a20a-6fc0fab92a1f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:01.019823 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 154/307] 10.222.0.23 () {56 vars in 1308 bytes} [Fri Sep 9 16:18:00 2022] GET /identity/v3/role_assignments?user.id=b8543bdc118f405d8bed154f2b5af95a&scope.system=all&role.id=17a32575480045789f8f279a3ada4fd1 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:01.089108 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:01.125163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943582c-5073-45ae-b0df-cb4129233174 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:01.162665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98f46b-847f-4823-8ff9-44d39a47087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:01.163462 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98f46b-847f-4823-8ff9-44d39a47087d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:01.163745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98f46b-847f-4823-8ff9-44d39a47087d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:01.164024 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98f46b-847f-4823-8ff9-44d39a47087d None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:01.164716 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98f46b-847f-4823-8ff9-44d39a47087d None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:01.165020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98f46b-847f-4823-8ff9-44d39a47087d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:01.166447 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98f46b-847f-4823-8ff9-44d39a47087d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:01.176311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 154/308] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:18:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:01.181955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89607a21-6920-4d7c-a20a-6fc0fab92a1f None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:01.231727 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:01.286792 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89607a21-6920-4d7c-a20a-6fc0fab92a1f None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:01.320965 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fdc011-2582-4d26-b5ee-809d6717c083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:01.321605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdc011-2582-4d26-b5ee-809d6717c083 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:01.321882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdc011-2582-4d26-b5ee-809d6717c083 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:01.322152 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdc011-2582-4d26-b5ee-809d6717c083 None admin] PATH_INFO: `/v3/users/30c9e9bf80b341df88707678dda9947c` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:01.323317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fdc011-2582-4d26-b5ee-809d6717c083 None admin] RBAC: Authorizing `identity:get_user(user_id=30c9e9bf80b341df88707678dda9947c)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:01.324772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fdc011-2582-4d26-b5ee-809d6717c083 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:01.326684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 155/309] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:18:01 2022] GET /identity/v3/users/30c9e9bf80b341df88707678dda9947c => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:01.336066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98f46b-847f-4823-8ff9-44d39a47087d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:01.442370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98f46b-847f-4823-8ff9-44d39a47087d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:01.472566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:01.478289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98cb48-07af-40e2-bb75-a3754ad1e15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:01.479255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98cb48-07af-40e2-bb75-a3754ad1e15d None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:01.479741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98cb48-07af-40e2-bb75-a3754ad1e15d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:01.480203 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98cb48-07af-40e2-bb75-a3754ad1e15d None admin] PATH_INFO: `/v3/system/users/30c9e9bf80b341df88707678dda9947c/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:01.481921 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98cb48-07af-40e2-bb75-a3754ad1e15d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=30c9e9bf80b341df88707678dda9947c, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:01.483561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98cb48-07af-40e2-bb75-a3754ad1e15d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:01.496397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 155/310] 10.222.0.23 () {58 vars in 1334 bytes} [Fri Sep 9 16:18:01 2022] PUT /identity/v3/system/users/30c9e9bf80b341df88707678dda9947c/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:18:01.842973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f74f24-1b0b-4c65-9eb1-40cc22e5b0ed None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:01.843364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f74f24-1b0b-4c65-9eb1-40cc22e5b0ed None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:01.843540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f74f24-1b0b-4c65-9eb1-40cc22e5b0ed None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:01.845151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 156/311] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:01.850476 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39fd0a-123a-47ce-9c08-d3468954dd1d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:01.850920 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39fd0a-123a-47ce-9c08-d3468954dd1d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:01.851195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39fd0a-123a-47ce-9c08-d3468954dd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:01.869402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-aa39fd0a-123a-47ce-9c08-d3468954dd1d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:01.870615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-aa39fd0a-123a-47ce-9c08-d3468954dd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:01.940485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 156/312] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:02.205972 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f74f24-1b0b-4c65-9eb1-40cc22e5b0ed None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:02.308325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f74f24-1b0b-4c65-9eb1-40cc22e5b0ed None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:02.322216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:02.342929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31d7f0-5861-42a3-a6e8-bef8ff5710fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:02.343568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31d7f0-5861-42a3-a6e8-bef8ff5710fd None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:02.343841 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31d7f0-5861-42a3-a6e8-bef8ff5710fd None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:02.344106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31d7f0-5861-42a3-a6e8-bef8ff5710fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:02.354290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31d7f0-5861-42a3-a6e8-bef8ff5710fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-cd31d7f0-5861-42a3-a6e8-bef8ff5710fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:02.359223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:02.360122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:02.360891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:02.360891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:02.360891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:02.360891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:02.360891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:02.360891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:02.360891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:02.360891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:02.360891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:02.360891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:02.360891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:18:02.360891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:02.360891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 157/313] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:02.368479 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39fd0a-123a-47ce-9c08-d3468954dd1d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:02.470696 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39fd0a-123a-47ce-9c08-d3468954dd1d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:02.496579 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:02.508053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaafa0e-421f-4ddd-8db5-0a2a2e5f1a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:02.508944 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaafa0e-421f-4ddd-8db5-0a2a2e5f1a20 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:02.509517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaafa0e-421f-4ddd-8db5-0a2a2e5f1a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:02.509971 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaafa0e-421f-4ddd-8db5-0a2a2e5f1a20 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:02.510844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaafa0e-421f-4ddd-8db5-0a2a2e5f1a20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:02.511350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaafa0e-421f-4ddd-8db5-0a2a2e5f1a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:02.512970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaafa0e-421f-4ddd-8db5-0a2a2e5f1a20 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:02.526290 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 157/314] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:18:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:02.531570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31d7f0-5861-42a3-a6e8-bef8ff5710fd None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:02.628808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31d7f0-5861-42a3-a6e8-bef8ff5710fd None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:02.663540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f14ad-d383-4a91-a743-a837ebcfa433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:02.664415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f14ad-d383-4a91-a743-a837ebcfa433 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:02.664831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f14ad-d383-4a91-a743-a837ebcfa433 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:02.665235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f14ad-d383-4a91-a743-a837ebcfa433 None admin] PATH_INFO: `/v3/users/52ad7f760538428abb568322ca4266da` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:02.666491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f14ad-d383-4a91-a743-a837ebcfa433 None admin] RBAC: Authorizing `identity:get_user(user_id=52ad7f760538428abb568322ca4266da)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:02.668216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f14ad-d383-4a91-a743-a837ebcfa433 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:02.670269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 158/315] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:18:02 2022] GET /identity/v3/users/52ad7f760538428abb568322ca4266da => generated 297 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:02.675835 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaafa0e-421f-4ddd-8db5-0a2a2e5f1a20 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:02.777758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaafa0e-421f-4ddd-8db5-0a2a2e5f1a20 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:02.811472 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060b135-4755-40c1-9800-a06c43761675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:02.812405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060b135-4755-40c1-9800-a06c43761675 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:02.812817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060b135-4755-40c1-9800-a06c43761675 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:02.813219 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060b135-4755-40c1-9800-a06c43761675 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:02.869013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:02.881254 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060b135-4755-40c1-9800-a06c43761675 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:02.882807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060b135-4755-40c1-9800-a06c43761675 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:02.987915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 158/316] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:02.993541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f14ad-d383-4a91-a743-a837ebcfa433 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:03.034654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:03.096575 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f14ad-d383-4a91-a743-a837ebcfa433 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:03.138267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ee1a9-ca55-4fef-970b-6e52a22e7c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:03.139384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ee1a9-ca55-4fef-970b-6e52a22e7c33 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:03.139861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ee1a9-ca55-4fef-970b-6e52a22e7c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:03.140336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ee1a9-ca55-4fef-970b-6e52a22e7c33 None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:03.141942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ee1a9-ca55-4fef-970b-6e52a22e7c33 None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:03.144049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ee1a9-ca55-4fef-970b-6e52a22e7c33 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:03.146594 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 159/317] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:02 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:03.155729 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060b135-4755-40c1-9800-a06c43761675 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:03.237400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc34b5-773d-4760-a5b8-ecfce34ec4e7 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:03.237935 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc34b5-773d-4760-a5b8-ecfce34ec4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:03.237935 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc34b5-773d-4760-a5b8-ecfce34ec4e7 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:03.239483 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 160/318] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:18:03.244500 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f404b-c1ec-4273-b76e-f7ebe237a4e7 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:03.244781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f404b-c1ec-4273-b76e-f7ebe237a4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:03.245051 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f404b-c1ec-4273-b76e-f7ebe237a4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:03.248878 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:03.264355 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d44f404b-c1ec-4273-b76e-f7ebe237a4e7 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:03.265081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060b135-4755-40c1-9800-a06c43761675 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:03.265526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d44f404b-c1ec-4273-b76e-f7ebe237a4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:03.301173 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d65ec2-9b4e-4155-a45d-e3b285a3ec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:03.301694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d65ec2-9b4e-4155-a45d-e3b285a3ec99 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:03.301969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d65ec2-9b4e-4155-a45d-e3b285a3ec99 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:03.302242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d65ec2-9b4e-4155-a45d-e3b285a3ec99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:03.303089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d65ec2-9b4e-4155-a45d-e3b285a3ec99 None admin] RBAC: Adding query filter params (role.id=17a32575480045789f8f279a3ada4fd1, scope.project.id=0b1e73957f8a494d88eab174721e9401, user.id=52ad7f760538428abb568322ca4266da) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:03.303368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d65ec2-9b4e-4155-a45d-e3b285a3ec99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:03.304902 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d65ec2-9b4e-4155-a45d-e3b285a3ec99 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:03.318413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 159/319] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:18:03 2022] GET /identity/v3/role_assignments?user.id=52ad7f760538428abb568322ca4266da&scope.project.id=0b1e73957f8a494d88eab174721e9401&role.id=17a32575480045789f8f279a3ada4fd1 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:03.318413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b62e3-478f-441c-8931-b6291f3c1b1b None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:03.318413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b62e3-478f-441c-8931-b6291f3c1b1b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:03.318413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b62e3-478f-441c-8931-b6291f3c1b1b None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:03.318413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 160/320] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:03.320666 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f2bf1-abab-41e3-9530-5910fc407c7d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:03.320947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f2bf1-abab-41e3-9530-5910fc407c7d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:03.321242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f2bf1-abab-41e3-9530-5910fc407c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:03.340824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a54f2bf1-abab-41e3-9530-5910fc407c7d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:03.342059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a54f2bf1-abab-41e3-9530-5910fc407c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:03.349845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 161/321] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:18:03.429533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 161/322] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:18:03.614763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f404b-c1ec-4273-b76e-f7ebe237a4e7 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:03.655323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:03.689914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f2bf1-abab-41e3-9530-5910fc407c7d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:03.718124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f404b-c1ec-4273-b76e-f7ebe237a4e7 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:03.750246 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc990d-dc78-4c74-8010-7b8befe610f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:03.750917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc990d-dc78-4c74-8010-7b8befe610f1 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:03.751186 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc990d-dc78-4c74-8010-7b8befe610f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:03.751475 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc990d-dc78-4c74-8010-7b8befe610f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:03.752557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc990d-dc78-4c74-8010-7b8befe610f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:03.753986 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc990d-dc78-4c74-8010-7b8befe610f1 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:03.755724 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 162/323] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:03.756816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9044b6-f935-4103-914c-1586e9f4c209 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:03.757091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9044b6-f935-4103-914c-1586e9f4c209 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:03.757358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9044b6-f935-4103-914c-1586e9f4c209 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:03.758668 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 163/324] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:03.760934 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9044b6-f935-4103-914c-1586e9f4c209 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:03.774416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:03.792761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f2bf1-abab-41e3-9530-5910fc407c7d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:03.826854 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa5691-5138-40d4-9a06-0de0cc4d49c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:03.827485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa5691-5138-40d4-9a06-0de0cc4d49c8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:03.827757 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa5691-5138-40d4-9a06-0de0cc4d49c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:03.828038 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa5691-5138-40d4-9a06-0de0cc4d49c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:03.838668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa5691-5138-40d4-9a06-0de0cc4d49c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-edaa5691-5138-40d4-9a06-0de0cc4d49c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:03.843442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:03.844388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:03.845186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:03.845186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:03.845186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:03.845186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:03.845186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:03.845186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:03.845186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:03.845186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:03.845186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:03.845186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:03.845186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:18:03.845186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:03.845186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 162/325] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:03.845901 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4dc84-7406-4621-833f-38d834e6f8fb None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:03.846179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4dc84-7406-4621-833f-38d834e6f8fb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:03.846438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4dc84-7406-4621-833f-38d834e6f8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:03.865015 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4ec4dc84-7406-4621-833f-38d834e6f8fb None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:03.865520 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9044b6-f935-4103-914c-1586e9f4c209 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:03.866120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4ec4dc84-7406-4621-833f-38d834e6f8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:03.873142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:03.899799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa45909-002b-4bb7-9b0a-8a895a3728bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:03.900460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa45909-002b-4bb7-9b0a-8a895a3728bd None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:03.900740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa45909-002b-4bb7-9b0a-8a895a3728bd None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:03.901006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa45909-002b-4bb7-9b0a-8a895a3728bd None admin] PATH_INFO: `/v3/groups` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:03.901639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa45909-002b-4bb7-9b0a-8a895a3728bd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:03.903103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa45909-002b-4bb7-9b0a-8a895a3728bd None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:03.919649 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 164/326] 10.222.0.23 () {60 vars in 1134 bytes} [Fri Sep 9 16:18:03 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:18:03.920632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa45909-002b-4bb7-9b0a-8a895a3728bd None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:03.955160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 163/327] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:04.029496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa45909-002b-4bb7-9b0a-8a895a3728bd None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:04.066102 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffba59-59b1-4ec2-9c41-4b691a1ce9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:04.067140 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffba59-59b1-4ec2-9c41-4b691a1ce9da None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:04.067396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffba59-59b1-4ec2-9c41-4b691a1ce9da None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:04.067686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffba59-59b1-4ec2-9c41-4b691a1ce9da None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:04.068373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dffba59-59b1-4ec2-9c41-4b691a1ce9da None admin] RBAC: Adding query filter params (name=reader) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:04.068676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dffba59-59b1-4ec2-9c41-4b691a1ce9da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:04.070165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dffba59-59b1-4ec2-9c41-4b691a1ce9da None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:04.083665 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 165/328] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:18:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:04.092199 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4dc84-7406-4621-833f-38d834e6f8fb None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:04.172386 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:04.202871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4dc84-7406-4621-833f-38d834e6f8fb None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:04.233178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffba59-59b1-4ec2-9c41-4b691a1ce9da None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:04.240674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d5cd7-4eb4-4054-8e41-5d8c4f33cdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:04.240674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d5cd7-4eb4-4054-8e41-5d8c4f33cdee None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:04.240674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d5cd7-4eb4-4054-8e41-5d8c4f33cdee None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:04.240970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d5cd7-4eb4-4054-8e41-5d8c4f33cdee None admin] PATH_INFO: `/v3/users/b8543bdc118f405d8bed154f2b5af95a` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:04.242008 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d5cd7-4eb4-4054-8e41-5d8c4f33cdee None admin] RBAC: Authorizing `identity:get_user(user_id=b8543bdc118f405d8bed154f2b5af95a)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:04.243575 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d5cd7-4eb4-4054-8e41-5d8c4f33cdee None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:04.245469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 164/329] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:18:04 2022] GET /identity/v3/users/b8543bdc118f405d8bed154f2b5af95a => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:04.251554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d5cd7-4eb4-4054-8e41-5d8c4f33cdee None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:04.271255 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:04.368551 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffba59-59b1-4ec2-9c41-4b691a1ce9da None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:04.372600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d5cd7-4eb4-4054-8e41-5d8c4f33cdee None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:04.413469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82af2e2-24bb-4efe-9bdb-df33db1689cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:04.414409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82af2e2-24bb-4efe-9bdb-df33db1689cf None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:04.414860 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82af2e2-24bb-4efe-9bdb-df33db1689cf None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:04.415295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82af2e2-24bb-4efe-9bdb-df33db1689cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:04.415403 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9e28e-4fbf-421a-a284-75ed2922f527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:04.416349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9e28e-4fbf-421a-a284-75ed2922f527 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:04.416713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9e28e-4fbf-421a-a284-75ed2922f527 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:04.417086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9e28e-4fbf-421a-a284-75ed2922f527 None admin] PATH_INFO: `/v3/system/users/b8543bdc118f405d8bed154f2b5af95a/roles/17a32575480045789f8f279a3ada4fd1` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:04.418620 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9e28e-4fbf-421a-a284-75ed2922f527 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b8543bdc118f405d8bed154f2b5af95a, role_id=17a32575480045789f8f279a3ada4fd1)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:04.420122 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9e28e-4fbf-421a-a284-75ed2922f527 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:04.429052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82af2e2-24bb-4efe-9bdb-df33db1689cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:04.432137 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 165/330] 10.222.0.23 () {58 vars in 1334 bytes} [Fri Sep 9 16:18:04 2022] PUT /identity/v3/system/users/b8543bdc118f405d8bed154f2b5af95a/roles/17a32575480045789f8f279a3ada4fd1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-a82af2e2-24bb-4efe-9bdb-df33db1689cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:04.439389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:04.440515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:04.441400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:04.441400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:04.441400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:04.441400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:04.441400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:04.441400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:04.441400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:04.441400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:04.441400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:04.441400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:04.441400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:04.441400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:04.443876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 166/331] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:04.449986 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9e28e-4fbf-421a-a284-75ed2922f527 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:04.454468 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:04.576152 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9e28e-4fbf-421a-a284-75ed2922f527 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:04.613492 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f640e46-65ec-42ab-8db5-db24b939a606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:04.614424 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f640e46-65ec-42ab-8db5-db24b939a606 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:04.614879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f640e46-65ec-42ab-8db5-db24b939a606 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:04.615294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f640e46-65ec-42ab-8db5-db24b939a606 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:04.616115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f640e46-65ec-42ab-8db5-db24b939a606 None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:04.616664 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f640e46-65ec-42ab-8db5-db24b939a606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:04.618240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f640e46-65ec-42ab-8db5-db24b939a606 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:04.631393 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 166/332] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:18:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:04.637543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82af2e2-24bb-4efe-9bdb-df33db1689cf None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:04.731514 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:04.745138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82af2e2-24bb-4efe-9bdb-df33db1689cf None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:04.788598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5271543-d648-42f4-b251-a7b00c29f618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:04.789385 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271543-d648-42f4-b251-a7b00c29f618 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:04.789730 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271543-d648-42f4-b251-a7b00c29f618 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:04.790063 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271543-d648-42f4-b251-a7b00c29f618 None admin] PATH_INFO: `/v3/users/30c9e9bf80b341df88707678dda9947c` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:04.792960 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5271543-d648-42f4-b251-a7b00c29f618 None admin] RBAC: Authorizing `identity:get_user(user_id=30c9e9bf80b341df88707678dda9947c)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:04.794512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5271543-d648-42f4-b251-a7b00c29f618 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:04.796487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 167/333] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:18:04 2022] GET /identity/v3/users/30c9e9bf80b341df88707678dda9947c => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:04.802328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f640e46-65ec-42ab-8db5-db24b939a606 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:04.893247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:04.939130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f640e46-65ec-42ab-8db5-db24b939a606 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:04.987857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799e7e0-2e47-45f7-a784-a6bae40b6749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:04.988395 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799e7e0-2e47-45f7-a784-a6bae40b6749 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:04.988671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799e7e0-2e47-45f7-a784-a6bae40b6749 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:04.988936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799e7e0-2e47-45f7-a784-a6bae40b6749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:04.989661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799e7e0-2e47-45f7-a784-a6bae40b6749 None admin] RBAC: Adding query filter params (role.id=9e02fa627e5c4388b0aa7182f1707895, user.id=30c9e9bf80b341df88707678dda9947c, scope.system=all) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:04.989967 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799e7e0-2e47-45f7-a784-a6bae40b6749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:04.991505 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799e7e0-2e47-45f7-a784-a6bae40b6749 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:05.006284 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 167/334] 10.222.0.23 () {56 vars in 1308 bytes} [Fri Sep 9 16:18:04 2022] GET /identity/v3/role_assignments?user.id=30c9e9bf80b341df88707678dda9947c&scope.system=all&role.id=9e02fa627e5c4388b0aa7182f1707895 => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:06.706693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fbee6-fa3b-47b5-8ead-100addd87a32 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:06.707717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fbee6-fa3b-47b5-8ead-100addd87a32 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:06.708217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fbee6-fa3b-47b5-8ead-100addd87a32 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:06.710133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 168/335] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:06.716188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab99c2b-5a05-4571-a428-d5ab99c8b4cf None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:06.716660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab99c2b-5a05-4571-a428-d5ab99c8b4cf None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:06.716759 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab99c2b-5a05-4571-a428-d5ab99c8b4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:06.743594 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-2ab99c2b-5a05-4571-a428-d5ab99c8b4cf None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:06.744814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2ab99c2b-5a05-4571-a428-d5ab99c8b4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:06.829714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 168/336] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:06.874039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828727e-9a7f-468d-a439-8762bf5db3d1 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:06.874373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828727e-9a7f-468d-a439-8762bf5db3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:06.874606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828727e-9a7f-468d-a439-8762bf5db3d1 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:06.876234 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 169/337] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:06.882112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daed37b-fa74-4b2a-bec0-0ccbe83cf0f8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:06.882403 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daed37b-fa74-4b2a-bec0-0ccbe83cf0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:06.882703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daed37b-fa74-4b2a-bec0-0ccbe83cf0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:06.905701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5daed37b-fa74-4b2a-bec0-0ccbe83cf0f8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:06.907546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5daed37b-fa74-4b2a-bec0-0ccbe83cf0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:06.965959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:06.986624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 169/338] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:18:07.180430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95f1a6-f189-4898-84b3-cfd08252aefa None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:07.180781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95f1a6-f189-4898-84b3-cfd08252aefa None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:07.180966 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95f1a6-f189-4898-84b3-cfd08252aefa None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:07.182654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 170/339] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:07.188763 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d886-7d4f-4ad9-8786-bfe1cd691057 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:07.189113 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d886-7d4f-4ad9-8786-bfe1cd691057 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:07.189321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d886-7d4f-4ad9-8786-bfe1cd691057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:07.211100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-dca1d886-7d4f-4ad9-8786-bfe1cd691057 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:07.212284 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dca1d886-7d4f-4ad9-8786-bfe1cd691057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:07.285608 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95f1a6-f189-4898-84b3-cfd08252aefa None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:07.301026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 170/340] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:07.301942 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1d886-7d4f-4ad9-8786-bfe1cd691057 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:07.412132 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95f1a6-f189-4898-84b3-cfd08252aefa None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:07.448896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc63035-5b68-478d-9df7-c0284d12008b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:07.449540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc63035-5b68-478d-9df7-c0284d12008b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:07.449815 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc63035-5b68-478d-9df7-c0284d12008b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:07.450085 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc63035-5b68-478d-9df7-c0284d12008b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:07.451317 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1d886-7d4f-4ad9-8786-bfe1cd691057 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:07.461494 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc63035-5b68-478d-9df7-c0284d12008b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-3cc63035-5b68-478d-9df7-c0284d12008b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:07.466657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:07.467542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:07.473369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:07.473369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:07.473369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:07.473369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:07.473369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:07.473369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:07.473369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:07.473369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:07.473369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:07.473369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:07.473369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:07.473369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:07.473369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 171/341] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:07.473369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40e8e4-2698-4cdb-96d5-ec1a2d6e0749 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:07.474084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40e8e4-2698-4cdb-96d5-ec1a2d6e0749 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:07.474084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40e8e4-2698-4cdb-96d5-ec1a2d6e0749 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:07.474084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 172/342] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:07.477364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a0575-3859-4650-acc9-1aba325507b7 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:07.477645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a0575-3859-4650-acc9-1aba325507b7 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:07.477913 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a0575-3859-4650-acc9-1aba325507b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:07.480849 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:07.497948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-2b0a0575-3859-4650-acc9-1aba325507b7 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:07.499179 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2b0a0575-3859-4650-acc9-1aba325507b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:07.510748 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:07.518273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a22b0-ab5b-4feb-8ef6-d7412950eb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:07.519305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a22b0-ab5b-4feb-8ef6-d7412950eb8a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:07.519778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a22b0-ab5b-4feb-8ef6-d7412950eb8a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:07.520312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a22b0-ab5b-4feb-8ef6-d7412950eb8a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:07.535723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a22b0-ab5b-4feb-8ef6-d7412950eb8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-b80a22b0-ab5b-4feb-8ef6-d7412950eb8a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:07.542951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:07.543865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:07.544633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:07.544633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:07.544633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:07.544633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:07.544633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:07.544633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:07.544633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:07.544633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:07.544633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:07.544633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:07.544633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:18:07.544633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:07.547959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 171/343] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:18:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:07.549599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a22b0-ab5b-4feb-8ef6-d7412950eb8a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:07.586639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 173/344] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:18:07.587630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a0575-3859-4650-acc9-1aba325507b7 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:07.697170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a22b0-ab5b-4feb-8ef6-d7412950eb8a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:07.729885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a0575-3859-4650-acc9-1aba325507b7 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:07.747755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9998f2-d056-45ea-a612-736acdae24cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:07.748390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9998f2-d056-45ea-a612-736acdae24cb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:07.748639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9998f2-d056-45ea-a612-736acdae24cb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:07.748913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9998f2-d056-45ea-a612-736acdae24cb None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:07.749570 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9998f2-d056-45ea-a612-736acdae24cb None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:07.749865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9998f2-d056-45ea-a612-736acdae24cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:07.751272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9998f2-d056-45ea-a612-736acdae24cb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:07.764436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 172/345] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:18:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:07.770072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9998f2-d056-45ea-a612-736acdae24cb None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:07.770617 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3af72e-cab0-467e-9585-9cbe256aad70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:07.771478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3af72e-cab0-467e-9585-9cbe256aad70 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:07.771947 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3af72e-cab0-467e-9585-9cbe256aad70 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:07.772804 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3af72e-cab0-467e-9585-9cbe256aad70 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:07.774276 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3af72e-cab0-467e-9585-9cbe256aad70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:07.774881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3af72e-cab0-467e-9585-9cbe256aad70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:07.776548 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3af72e-cab0-467e-9585-9cbe256aad70 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:07.786242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 174/346] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:07.791413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3af72e-cab0-467e-9585-9cbe256aad70 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:07.815986 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:07.833714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:07.919392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9998f2-d056-45ea-a612-736acdae24cb None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:07.984520 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3af72e-cab0-467e-9585-9cbe256aad70 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:08.012308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8bdd9-5be8-4b26-83c8-aa2137f36904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:08.012841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8bdd9-5be8-4b26-83c8-aa2137f36904 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:08.013113 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8bdd9-5be8-4b26-83c8-aa2137f36904 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:08.013375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8bdd9-5be8-4b26-83c8-aa2137f36904 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:08.021830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5c24a-1cc8-4f7c-a27f-0a09d328b4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:08.022889 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5c24a-1cc8-4f7c-a27f-0a09d328b4fa None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:08.023361 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5c24a-1cc8-4f7c-a27f-0a09d328b4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:08.023856 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5c24a-1cc8-4f7c-a27f-0a09d328b4fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:08.105576 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8bdd9-5be8-4b26-83c8-aa2137f36904 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:08.106859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5c24a-1cc8-4f7c-a27f-0a09d328b4fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:08.107237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8bdd9-5be8-4b26-83c8-aa2137f36904 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:08.108786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5c24a-1cc8-4f7c-a27f-0a09d328b4fa None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:08.121757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:08.152879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:08.225233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 175/347] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:08.226141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5c24a-1cc8-4f7c-a27f-0a09d328b4fa None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:08.236540 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 173/348] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 16:18:08.237736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8bdd9-5be8-4b26-83c8-aa2137f36904 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:08.362888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5c24a-1cc8-4f7c-a27f-0a09d328b4fa None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:08.374061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:08.379235 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8bdd9-5be8-4b26-83c8-aa2137f36904 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:08.403161 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4e9df-9fa0-4aec-a4a2-43e7504405bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:08.403802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4e9df-9fa0-4aec-a4a2-43e7504405bc None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:08.404081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4e9df-9fa0-4aec-a4a2-43e7504405bc None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:08.404344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4e9df-9fa0-4aec-a4a2-43e7504405bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:08.419150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4e9df-9fa0-4aec-a4a2-43e7504405bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-a4e4e9df-9fa0-4aec-a4a2-43e7504405bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:08.424006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:08.424921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:08.426123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:08.426123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:08.426123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:08.426123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:08.426123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:08.426123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:08.426123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:08.426123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:08.426123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:08.426123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:08.426123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:08.426123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:08.426123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 176/349] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:08.426123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4e9df-9fa0-4aec-a4a2-43e7504405bc None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:08.435185 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030655d-1c46-4971-9687-f189735ad1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:08.435185 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030655d-1c46-4971-9687-f189735ad1fb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:08.435446 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030655d-1c46-4971-9687-f189735ad1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:08.435648 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030655d-1c46-4971-9687-f189735ad1fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:08.439686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:08.449799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030655d-1c46-4971-9687-f189735ad1fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-4030655d-1c46-4971-9687-f189735ad1fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:08.455796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:08.456880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:08.457876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:08.457876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:08.457876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:08.457876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:08.457876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:08.457876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:08.457876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:08.457876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:08.457876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:08.457876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:08.457876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 16:18:08.457876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:08.459516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 174/350] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:08.460651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030655d-1c46-4971-9687-f189735ad1fb None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:08.561495 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4e9df-9fa0-4aec-a4a2-43e7504405bc None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:08.582429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030655d-1c46-4971-9687-f189735ad1fb None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:08.602488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24012280-0bb0-4145-8684-b777c494f338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:08.603443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24012280-0bb0-4145-8684-b777c494f338 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:08.603857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24012280-0bb0-4145-8684-b777c494f338 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:08.604264 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24012280-0bb0-4145-8684-b777c494f338 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:08.605550 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24012280-0bb0-4145-8684-b777c494f338 None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:08.607202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24012280-0bb0-4145-8684-b777c494f338 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:08.610029 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 177/351] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:08 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:08.611290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24012280-0bb0-4145-8684-b777c494f338 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:08.620488 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de38a4a-d7fe-4dd7-b082-edfee03677a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:08.621052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de38a4a-d7fe-4dd7-b082-edfee03677a6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:08.621372 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de38a4a-d7fe-4dd7-b082-edfee03677a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:08.621642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de38a4a-d7fe-4dd7-b082-edfee03677a6 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:08.622890 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de38a4a-d7fe-4dd7-b082-edfee03677a6 None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:08.624315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de38a4a-d7fe-4dd7-b082-edfee03677a6 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:08.626028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 175/352] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:08 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:08.626932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de38a4a-d7fe-4dd7-b082-edfee03677a6 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:08.654373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:08.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:08.741082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24012280-0bb0-4145-8684-b777c494f338 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:08.754335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de38a4a-d7fe-4dd7-b082-edfee03677a6 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:08.784117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487c48f-fe7a-4ad9-af74-2c85cc35b855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:08.785105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487c48f-fe7a-4ad9-af74-2c85cc35b855 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:08.785521 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487c48f-fe7a-4ad9-af74-2c85cc35b855 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:08.785999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487c48f-fe7a-4ad9-af74-2c85cc35b855 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:08.786825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487c48f-fe7a-4ad9-af74-2c85cc35b855 None admin] RBAC: Adding query filter params (name=member) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:08.787358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487c48f-fe7a-4ad9-af74-2c85cc35b855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:08.788997 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487c48f-fe7a-4ad9-af74-2c85cc35b855 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:08.790661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9a12a-380e-4277-a478-42efad5979f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:08.791274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9a12a-380e-4277-a478-42efad5979f2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:08.791550 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9a12a-380e-4277-a478-42efad5979f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:08.791816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9a12a-380e-4277-a478-42efad5979f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:08.792480 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9a12a-380e-4277-a478-42efad5979f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:08.792779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9a12a-380e-4277-a478-42efad5979f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:08.794268 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9a12a-380e-4277-a478-42efad5979f2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:08.799840 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 178/353] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:18:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:08.801656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487c48f-fe7a-4ad9-af74-2c85cc35b855 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:08.804481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 176/354] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:18:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:08.805454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9a12a-380e-4277-a478-42efad5979f2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:08.920007 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:08.924962 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487c48f-fe7a-4ad9-af74-2c85cc35b855 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:08.926966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9a12a-380e-4277-a478-42efad5979f2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:08.959295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:08.970195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f03e9-8ca7-4bab-b694-9264c2ac6a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:08.970923 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e628ed-c1bf-4a35-8381-b65d3dca5c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:08.971284 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f03e9-8ca7-4bab-b694-9264c2ac6a34 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:08.971702 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f03e9-8ca7-4bab-b694-9264c2ac6a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:08.971811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e628ed-c1bf-4a35-8381-b65d3dca5c4e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:08.972160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e628ed-c1bf-4a35-8381-b65d3dca5c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:08.972350 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f03e9-8ca7-4bab-b694-9264c2ac6a34 None admin] PATH_INFO: `/v3/groups/cfed87f511354222848ae0fb14591553` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:08.972592 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e628ed-c1bf-4a35-8381-b65d3dca5c4e None admin] PATH_INFO: `/v3/groups/cfed87f511354222848ae0fb14591553` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:08.979422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e628ed-c1bf-4a35-8381-b65d3dca5c4e None admin] RBAC: Authorizing `identity:get_group(group_id=cfed87f511354222848ae0fb14591553)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:08.980453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f03e9-8ca7-4bab-b694-9264c2ac6a34 None admin] RBAC: Authorizing `identity:get_group(group_id=cfed87f511354222848ae0fb14591553)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:08.981314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e628ed-c1bf-4a35-8381-b65d3dca5c4e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:08.982230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f03e9-8ca7-4bab-b694-9264c2ac6a34 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:08.983241 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 177/355] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:08 2022] GET /identity/v3/groups/cfed87f511354222848ae0fb14591553 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:08.984082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e628ed-c1bf-4a35-8381-b65d3dca5c4e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:08.984745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 179/356] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:08 2022] GET /identity/v3/groups/cfed87f511354222848ae0fb14591553 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:08.985659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f03e9-8ca7-4bab-b694-9264c2ac6a34 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:09.107375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e628ed-c1bf-4a35-8381-b65d3dca5c4e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:09.114853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f03e9-8ca7-4bab-b694-9264c2ac6a34 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:09.156399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa650f4-e2bb-4aa0-ab59-d34fd2ea27dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:09.156651 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be6a9c-f197-43d8-a041-a6d75a6490c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:09.157076 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa650f4-e2bb-4aa0-ab59-d34fd2ea27dd None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:09.157355 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa650f4-e2bb-4aa0-ab59-d34fd2ea27dd None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:09.157628 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa650f4-e2bb-4aa0-ab59-d34fd2ea27dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:09.157995 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be6a9c-f197-43d8-a041-a6d75a6490c8 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:09.158274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be6a9c-f197-43d8-a041-a6d75a6490c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:09.158586 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be6a9c-f197-43d8-a041-a6d75a6490c8 None admin] PATH_INFO: `/v3/users/b8543bdc118f405d8bed154f2b5af95a` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:09.159854 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be6a9c-f197-43d8-a041-a6d75a6490c8 None admin] RBAC: Authorizing `identity:get_user(user_id=b8543bdc118f405d8bed154f2b5af95a)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:09.161373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be6a9c-f197-43d8-a041-a6d75a6490c8 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:09.163421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 180/357] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:18:08 2022] GET /identity/v3/users/b8543bdc118f405d8bed154f2b5af95a => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:09.164007 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be6a9c-f197-43d8-a041-a6d75a6490c8 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:09.199010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:09.245186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:09.258977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa650f4-e2bb-4aa0-ab59-d34fd2ea27dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:09.261397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa650f4-e2bb-4aa0-ab59-d34fd2ea27dd None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:09.286912 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be6a9c-f197-43d8-a041-a6d75a6490c8 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:09.325636 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49c9a4-c5d8-4506-8bc6-3414a5107395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:09.326267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49c9a4-c5d8-4506-8bc6-3414a5107395 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:09.326583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49c9a4-c5d8-4506-8bc6-3414a5107395 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:09.326835 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49c9a4-c5d8-4506-8bc6-3414a5107395 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:09.327566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49c9a4-c5d8-4506-8bc6-3414a5107395 None admin] RBAC: Adding query filter params (group.id=cfed87f511354222848ae0fb14591553, role.id=9e02fa627e5c4388b0aa7182f1707895, scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:09.327871 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49c9a4-c5d8-4506-8bc6-3414a5107395 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:09.329517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49c9a4-c5d8-4506-8bc6-3414a5107395 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:09.337042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 181/358] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Sep 9 16:18:09 2022] GET /identity/v3/role_assignments?group.id=cfed87f511354222848ae0fb14591553&scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b&role.id=9e02fa627e5c4388b0aa7182f1707895 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:09.337190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49c9a4-c5d8-4506-8bc6-3414a5107395 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:09.379515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 178/359] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:09.381565 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5aa71-6d01-40b4-b625-c0ad28806a51 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:09.382384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5aa71-6d01-40b4-b625-c0ad28806a51 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:09.382881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5aa71-6d01-40b4-b625-c0ad28806a51 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:09.384535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 179/360] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:09.385107 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5aa71-6d01-40b4-b625-c0ad28806a51 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:09.475345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:09.483375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49c9a4-c5d8-4506-8bc6-3414a5107395 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:09.523115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:09.529210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84cce0-0ca8-49d8-a04b-94cb10458a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:09.530442 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84cce0-0ca8-49d8-a04b-94cb10458a50 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:09.530777 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84cce0-0ca8-49d8-a04b-94cb10458a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:09.531041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84cce0-0ca8-49d8-a04b-94cb10458a50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:09.531780 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84cce0-0ca8-49d8-a04b-94cb10458a50 None admin] RBAC: Adding query filter params (group.id=cfed87f511354222848ae0fb14591553, role.id=0d40b3176f78460f9c4a29c1e9ef843e, scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:09.532093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84cce0-0ca8-49d8-a04b-94cb10458a50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:09.534726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84cce0-0ca8-49d8-a04b-94cb10458a50 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:09.539141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5aa71-6d01-40b4-b625-c0ad28806a51 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:09.541217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 182/361] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Sep 9 16:18:09 2022] GET /identity/v3/role_assignments?group.id=cfed87f511354222848ae0fb14591553&scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b&role.id=0d40b3176f78460f9c4a29c1e9ef843e => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:09.542432 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbc4a8-42c9-4d1f-a54c-bb51cf4b7800 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:09.542828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbc4a8-42c9-4d1f-a54c-bb51cf4b7800 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:09.543102 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbc4a8-42c9-4d1f-a54c-bb51cf4b7800 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:09.544318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 183/362] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:09.544997 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbc4a8-42c9-4d1f-a54c-bb51cf4b7800 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:09.597821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2bba1-29a4-4e21-ae85-4654d2a3a095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:09.599183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2bba1-29a4-4e21-ae85-4654d2a3a095 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:09.599660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2bba1-29a4-4e21-ae85-4654d2a3a095 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:09.600147 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2bba1-29a4-4e21-ae85-4654d2a3a095 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:09.601473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2bba1-29a4-4e21-ae85-4654d2a3a095 None admin] RBAC: Adding query filter params (role.id=17a32575480045789f8f279a3ada4fd1, user.id=b8543bdc118f405d8bed154f2b5af95a, scope.system=all) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:09.602186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2bba1-29a4-4e21-ae85-4654d2a3a095 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:09.604070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2bba1-29a4-4e21-ae85-4654d2a3a095 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:09.626532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 180/363] 10.222.0.23 () {56 vars in 1308 bytes} [Fri Sep 9 16:18:09 2022] GET /identity/v3/role_assignments?user.id=b8543bdc118f405d8bed154f2b5af95a&scope.system=all&role.id=17a32575480045789f8f279a3ada4fd1 => generated 510 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:09.626532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d9f4a-9a34-4d84-b33d-7669da55d8e5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:09.626532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d9f4a-9a34-4d84-b33d-7669da55d8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:09.626532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d9f4a-9a34-4d84-b33d-7669da55d8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:09.648962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f15d9f4a-9a34-4d84-b33d-7669da55d8e5 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:09.652568 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f15d9f4a-9a34-4d84-b33d-7669da55d8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:09.728285 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbc4a8-42c9-4d1f-a54c-bb51cf4b7800 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:09.781803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 181/364] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:09.786063 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afe88c-d401-428e-8c18-5d5a86c7960b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:09.786063 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afe88c-d401-428e-8c18-5d5a86c7960b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:09.786063 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afe88c-d401-428e-8c18-5d5a86c7960b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:09.798683 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b460a46-5d16-4c48-b30b-10f3e496fd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:09.800057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b460a46-5d16-4c48-b30b-10f3e496fd84 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:09.800498 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b460a46-5d16-4c48-b30b-10f3e496fd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:09.800901 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b460a46-5d16-4c48-b30b-10f3e496fd84 None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:09.803254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b460a46-5d16-4c48-b30b-10f3e496fd84 None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:09.805078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b460a46-5d16-4c48-b30b-10f3e496fd84 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:09.805834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-99afe88c-d401-428e-8c18-5d5a86c7960b None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:09.809353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-99afe88c-d401-428e-8c18-5d5a86c7960b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:09.812322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 184/365] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:09 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:09.826958 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b460a46-5d16-4c48-b30b-10f3e496fd84 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:09.897270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:09.926549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 182/366] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:10.014811 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b460a46-5d16-4c48-b30b-10f3e496fd84 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:10.052117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fd649-3f44-4509-bf0c-bf1331ad0e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:10.052769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fd649-3f44-4509-bf0c-bf1331ad0e85 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:10.053049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fd649-3f44-4509-bf0c-bf1331ad0e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:10.053323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fd649-3f44-4509-bf0c-bf1331ad0e85 None admin] PATH_INFO: `/v3/groups/cfed87f511354222848ae0fb14591553` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:10.054486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fd649-3f44-4509-bf0c-bf1331ad0e85 None admin] RBAC: Authorizing `identity:get_group(group_id=cfed87f511354222848ae0fb14591553)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:10.055970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fd649-3f44-4509-bf0c-bf1331ad0e85 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:10.057923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 185/367] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:09 2022] GET /identity/v3/groups/cfed87f511354222848ae0fb14591553 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:10.064032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afe88c-d401-428e-8c18-5d5a86c7960b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:10.094887 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:10.163435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fd649-3f44-4509-bf0c-bf1331ad0e85 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:10.250211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afe88c-d401-428e-8c18-5d5a86c7960b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:10.326699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9598a-7b40-4697-8e44-71f0e29becb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:10.327245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9598a-7b40-4697-8e44-71f0e29becb9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:10.327522 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9598a-7b40-4697-8e44-71f0e29becb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:10.327794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9598a-7b40-4697-8e44-71f0e29becb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:10.328523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9598a-7b40-4697-8e44-71f0e29becb9 None admin] RBAC: Adding query filter params (group.id=cfed87f511354222848ae0fb14591553, role.id=9e02fa627e5c4388b0aa7182f1707895, scope.project.id=0b1e73957f8a494d88eab174721e9401) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:10.328828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9598a-7b40-4697-8e44-71f0e29becb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:10.330266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9598a-7b40-4697-8e44-71f0e29becb9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:10.337194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 183/368] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Sep 9 16:18:10 2022] GET /identity/v3/role_assignments?group.id=cfed87f511354222848ae0fb14591553&scope.project.id=0b1e73957f8a494d88eab174721e9401&role.id=9e02fa627e5c4388b0aa7182f1707895 => generated 260 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:10.338033 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9598a-7b40-4697-8e44-71f0e29becb9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:10.350309 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:10.358556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fd649-3f44-4509-bf0c-bf1331ad0e85 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:10.430578 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ef2a4-9411-4c57-94c1-8fe36cdbb4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:10.431252 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ef2a4-9411-4c57-94c1-8fe36cdbb4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:10.431580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ef2a4-9411-4c57-94c1-8fe36cdbb4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:10.431795 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ef2a4-9411-4c57-94c1-8fe36cdbb4e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:10.458475 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ef2a4-9411-4c57-94c1-8fe36cdbb4e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-d12ef2a4-9411-4c57-94c1-8fe36cdbb4e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:10.471146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:10.472181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:10.473467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:10.473467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:10.473467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:10.473467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:10.473467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:10.473467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:10.473467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:10.473467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:10.473467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:10.473467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:10.473467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:18:10.473467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:10.474115 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 186/369] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:18:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:10.482450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ef2a4-9411-4c57-94c1-8fe36cdbb4e6 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:10.486568 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:10.551567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9598a-7b40-4697-8e44-71f0e29becb9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:10.625668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6f1e0-c1cc-433b-8676-3c03f7059693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:10.626320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6f1e0-c1cc-433b-8676-3c03f7059693 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:10.626634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6f1e0-c1cc-433b-8676-3c03f7059693 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:10.626900 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6f1e0-c1cc-433b-8676-3c03f7059693 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:10.653628 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6f1e0-c1cc-433b-8676-3c03f7059693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:10.658697 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ef2a4-9411-4c57-94c1-8fe36cdbb4e6 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-98c6f1e0-c1cc-433b-8676-3c03f7059693 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:10.659018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:10.661081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:10.662639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:10.662639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:10.662639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:10.662639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:10.662639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:10.662639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:10.662639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:10.662639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:10.662639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:10.662639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:10.662639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:18:10.662639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:10.662639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 184/370] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:18:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:10.669345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6f1e0-c1cc-433b-8676-3c03f7059693 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:10.702311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df64d4-6df0-4256-9666-6eb7c36f2740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:10.702980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df64d4-6df0-4256-9666-6eb7c36f2740 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:10.703299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df64d4-6df0-4256-9666-6eb7c36f2740 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:10.703571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df64d4-6df0-4256-9666-6eb7c36f2740 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:10.704249 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df64d4-6df0-4256-9666-6eb7c36f2740 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:10.704552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df64d4-6df0-4256-9666-6eb7c36f2740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:10.705996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df64d4-6df0-4256-9666-6eb7c36f2740 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:10.715902 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 187/371] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:10.722172 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df64d4-6df0-4256-9666-6eb7c36f2740 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:10.745121 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:10.887340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df64d4-6df0-4256-9666-6eb7c36f2740 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:10.908224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6f1e0-c1cc-433b-8676-3c03f7059693 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:10.954561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:10.959191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d27565-3dd0-41ce-b1f7-b6668622fca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:10.959805 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d27565-3dd0-41ce-b1f7-b6668622fca1 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:10.960089 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d27565-3dd0-41ce-b1f7-b6668622fca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:10.960360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d27565-3dd0-41ce-b1f7-b6668622fca1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:10.961649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec679e3-305f-4da3-a210-608a68bf2c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:10.965488 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec679e3-305f-4da3-a210-608a68bf2c07 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:10.966198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec679e3-305f-4da3-a210-608a68bf2c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:10.966470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec679e3-305f-4da3-a210-608a68bf2c07 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:10.967175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec679e3-305f-4da3-a210-608a68bf2c07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:10.967476 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec679e3-305f-4da3-a210-608a68bf2c07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:10.969196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec679e3-305f-4da3-a210-608a68bf2c07 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:10.983532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 185/372] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:10.989645 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec679e3-305f-4da3-a210-608a68bf2c07 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:11.116148 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec679e3-305f-4da3-a210-608a68bf2c07 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:11.125989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d27565-3dd0-41ce-b1f7-b6668622fca1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:11.127497 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d27565-3dd0-41ce-b1f7-b6668622fca1 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:11.156810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:11.180771 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681904ed-de97-448f-b975-4119aeabaf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:11.181386 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681904ed-de97-448f-b975-4119aeabaf0b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:11.181660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681904ed-de97-448f-b975-4119aeabaf0b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:11.181924 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681904ed-de97-448f-b975-4119aeabaf0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:11.293959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:11.305937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 188/373] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 16:18:11.307123 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681904ed-de97-448f-b975-4119aeabaf0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:11.308671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681904ed-de97-448f-b975-4119aeabaf0b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:11.313076 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d27565-3dd0-41ce-b1f7-b6668622fca1 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:11.471519 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d27565-3dd0-41ce-b1f7-b6668622fca1 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:11.496595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 186/374] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 16:18:11.503772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681904ed-de97-448f-b975-4119aeabaf0b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:11.507595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:11.545758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dbd9e-6398-47bd-bc0c-5e449b7d886d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:11.546433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dbd9e-6398-47bd-bc0c-5e449b7d886d None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:11.546756 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dbd9e-6398-47bd-bc0c-5e449b7d886d None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:11.547030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dbd9e-6398-47bd-bc0c-5e449b7d886d None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:11.548214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dbd9e-6398-47bd-bc0c-5e449b7d886d None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:11.549712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dbd9e-6398-47bd-bc0c-5e449b7d886d None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:11.553003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 189/375] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:11 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:11.557806 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dbd9e-6398-47bd-bc0c-5e449b7d886d None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:11.662865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:11.671779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681904ed-de97-448f-b975-4119aeabaf0b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:11.741274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55121085-aa93-4b48-b341-69b4fb2c20d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:11.741858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55121085-aa93-4b48-b341-69b4fb2c20d0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:11.742133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55121085-aa93-4b48-b341-69b4fb2c20d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:11.742404 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55121085-aa93-4b48-b341-69b4fb2c20d0 None admin] PATH_INFO: `/v3/projects/f5e628b418b44cb0b8fcf5460ded8ac8` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:11.745059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55121085-aa93-4b48-b341-69b4fb2c20d0 None admin] RBAC: Authorizing `identity:get_project(project_id=f5e628b418b44cb0b8fcf5460ded8ac8)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:11.746544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55121085-aa93-4b48-b341-69b4fb2c20d0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:11.749383 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 187/376] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:11 2022] GET /identity/v3/projects/f5e628b418b44cb0b8fcf5460ded8ac8 => generated 344 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:11.755489 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55121085-aa93-4b48-b341-69b4fb2c20d0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:11.771111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dbd9e-6398-47bd-bc0c-5e449b7d886d None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:11.836470 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf87df-02bd-4b40-85a2-f7327f388e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:11.837122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf87df-02bd-4b40-85a2-f7327f388e55 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:11.837398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf87df-02bd-4b40-85a2-f7327f388e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:11.837667 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf87df-02bd-4b40-85a2-f7327f388e55 None admin] PATH_INFO: `/v3/groups/cfed87f511354222848ae0fb14591553` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:11.839120 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf87df-02bd-4b40-85a2-f7327f388e55 None admin] RBAC: Authorizing `identity:get_group(group_id=cfed87f511354222848ae0fb14591553)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:11.840566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf87df-02bd-4b40-85a2-f7327f388e55 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:11.843030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 190/377] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:11 2022] GET /identity/v3/groups/cfed87f511354222848ae0fb14591553 => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:11.852541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379387f3-0d13-4f7e-a98b-f433a1968e95 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:11.852831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379387f3-0d13-4f7e-a98b-f433a1968e95 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:11.853086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379387f3-0d13-4f7e-a98b-f433a1968e95 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:11.854922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 191/378] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:11.855622 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379387f3-0d13-4f7e-a98b-f433a1968e95 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:11.932761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55121085-aa93-4b48-b341-69b4fb2c20d0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:11.934095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:11.984957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e276f-91d8-40bd-b2c0-4f99f4936882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:11.985994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e276f-91d8-40bd-b2c0-4f99f4936882 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:11.986459 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e276f-91d8-40bd-b2c0-4f99f4936882 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:11.986950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e276f-91d8-40bd-b2c0-4f99f4936882 None admin] PATH_INFO: `/v3/groups/b771303ebdf949e7861d16757a60e95f` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:11.995843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e276f-91d8-40bd-b2c0-4f99f4936882 None admin] RBAC: Authorizing `identity:get_group(group_id=b771303ebdf949e7861d16757a60e95f)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:11.997589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e276f-91d8-40bd-b2c0-4f99f4936882 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:11.999758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 188/379] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:11 2022] GET /identity/v3/groups/b771303ebdf949e7861d16757a60e95f => generated 229 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:12.001132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362efcce-e8f9-4786-8983-68e26bfbcf9f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:12.001607 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362efcce-e8f9-4786-8983-68e26bfbcf9f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:12.002072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362efcce-e8f9-4786-8983-68e26bfbcf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:12.005615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379387f3-0d13-4f7e-a98b-f433a1968e95 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:12.025679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-362efcce-e8f9-4786-8983-68e26bfbcf9f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:12.027244 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-362efcce-e8f9-4786-8983-68e26bfbcf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:12.048803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:12.057347 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266ae78-21dc-483b-ae52-071d600abffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:12.058290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266ae78-21dc-483b-ae52-071d600abffb None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:12.058794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266ae78-21dc-483b-ae52-071d600abffb None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:12.059387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266ae78-21dc-483b-ae52-071d600abffb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:12.060345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266ae78-21dc-483b-ae52-071d600abffb None admin] RBAC: Adding query filter params (group.id=cfed87f511354222848ae0fb14591553, role.id=0d40b3176f78460f9c4a29c1e9ef843e, scope.project.id=0b1e73957f8a494d88eab174721e9401) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:12.061265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266ae78-21dc-483b-ae52-071d600abffb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:12.062984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266ae78-21dc-483b-ae52-071d600abffb None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:12.070898 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 192/380] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Sep 9 16:18:11 2022] GET /identity/v3/role_assignments?group.id=cfed87f511354222848ae0fb14591553&scope.project.id=0b1e73957f8a494d88eab174721e9401&role.id=0d40b3176f78460f9c4a29c1e9ef843e => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:12.071568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266ae78-21dc-483b-ae52-071d600abffb None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:12.133615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 189/381] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:12.187134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266ae78-21dc-483b-ae52-071d600abffb None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:12.205759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:12.225062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322b214-f5c1-46eb-87c8-5a6abb4b2889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:12.225759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322b214-f5c1-46eb-87c8-5a6abb4b2889 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:12.226033 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322b214-f5c1-46eb-87c8-5a6abb4b2889 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:12.226297 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322b214-f5c1-46eb-87c8-5a6abb4b2889 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:12.227061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322b214-f5c1-46eb-87c8-5a6abb4b2889 None admin] RBAC: Adding query filter params (group.id=b771303ebdf949e7861d16757a60e95f, role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=f5e628b418b44cb0b8fcf5460ded8ac8) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:12.227363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322b214-f5c1-46eb-87c8-5a6abb4b2889 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:12.228813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322b214-f5c1-46eb-87c8-5a6abb4b2889 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:12.235243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 193/382] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Sep 9 16:18:12 2022] GET /identity/v3/role_assignments?group.id=b771303ebdf949e7861d16757a60e95f&scope.project.id=f5e628b418b44cb0b8fcf5460ded8ac8&role.id=c6da7ba2be504943af29338ee8855f05 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:12.381527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af165068-0028-4cbb-8870-d299c9b6eb18 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:12.382309 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af165068-0028-4cbb-8870-d299c9b6eb18 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:12.382843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af165068-0028-4cbb-8870-d299c9b6eb18 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:12.384746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 190/383] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:12.390090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e6332-7bee-4547-abb9-65f794592250 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:12.390374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e6332-7bee-4547-abb9-65f794592250 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:12.390674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e6332-7bee-4547-abb9-65f794592250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:12.419699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-9a4e6332-7bee-4547-abb9-65f794592250 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:12.420948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-9a4e6332-7bee-4547-abb9-65f794592250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:12.497978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 194/384] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:12.504922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af165068-0028-4cbb-8870-d299c9b6eb18 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:12.636255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af165068-0028-4cbb-8870-d299c9b6eb18 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:12.679284 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799a83f-fbee-4a53-bab7-ffead25696df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:12.679988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799a83f-fbee-4a53-bab7-ffead25696df None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:12.680265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799a83f-fbee-4a53-bab7-ffead25696df None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:12.680530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799a83f-fbee-4a53-bab7-ffead25696df None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:12.693239 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799a83f-fbee-4a53-bab7-ffead25696df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-2799a83f-fbee-4a53-bab7-ffead25696df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:12.698150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:12.699041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:12.699828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:12.699828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:12.699828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:12.699828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:12.699828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:12.699828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:12.699828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:12.699828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:12.699828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:12.699828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:12.699828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:12.699828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:12.699828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 191/385] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:12.705600 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e6332-7bee-4547-abb9-65f794592250 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:12.820023 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799a83f-fbee-4a53-bab7-ffead25696df None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:12.831669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e6332-7bee-4547-abb9-65f794592250 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:12.845477 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:12.864854 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:12.892287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f300e5-0a32-4d3b-aee8-95bee1f4775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:12.892838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f300e5-0a32-4d3b-aee8-95bee1f4775b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:12.893120 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f300e5-0a32-4d3b-aee8-95bee1f4775b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:12.893394 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f300e5-0a32-4d3b-aee8-95bee1f4775b None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:12.894083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f300e5-0a32-4d3b-aee8-95bee1f4775b None admin] RBAC: Adding query filter params (name=member) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:12.894374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f300e5-0a32-4d3b-aee8-95bee1f4775b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:12.895790 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f300e5-0a32-4d3b-aee8-95bee1f4775b None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:12.910445 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 195/386] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:18:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:12.920615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f300e5-0a32-4d3b-aee8-95bee1f4775b None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:12.954540 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799a83f-fbee-4a53-bab7-ffead25696df None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:12.996109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6587e-95fd-4d67-8b70-3d1d37def090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:12.996753 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6587e-95fd-4d67-8b70-3d1d37def090 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:12.997031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6587e-95fd-4d67-8b70-3d1d37def090 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:12.997304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6587e-95fd-4d67-8b70-3d1d37def090 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:13.012261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6587e-95fd-4d67-8b70-3d1d37def090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-e2a6587e-95fd-4d67-8b70-3d1d37def090 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:13.019337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:13.020306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:13.021080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:13.021080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:13.021080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:13.021080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:13.021080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:13.021080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:13.021080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:13.021080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:13.021080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:13.021080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:13.021080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:18:13.021080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:13.021080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 192/387] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:18:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:13.030084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6587e-95fd-4d67-8b70-3d1d37def090 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:13.151150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f300e5-0a32-4d3b-aee8-95bee1f4775b None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:13.188876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:13.224788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a773b3-d811-4307-99bf-af7f5ad5284c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:13.225533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a773b3-d811-4307-99bf-af7f5ad5284c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:13.225810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a773b3-d811-4307-99bf-af7f5ad5284c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:13.226078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a773b3-d811-4307-99bf-af7f5ad5284c None admin] PATH_INFO: `/v3/groups/cfed87f511354222848ae0fb14591553` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:13.227424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a773b3-d811-4307-99bf-af7f5ad5284c None admin] RBAC: Authorizing `identity:get_group(group_id=cfed87f511354222848ae0fb14591553)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:13.228935 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a773b3-d811-4307-99bf-af7f5ad5284c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:13.230911 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 196/388] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:12 2022] GET /identity/v3/groups/cfed87f511354222848ae0fb14591553 => generated 226 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:13.243660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6587e-95fd-4d67-8b70-3d1d37def090 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:13.247714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a773b3-d811-4307-99bf-af7f5ad5284c None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:13.313619 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd416304-bb12-4158-ac85-4fe1426501fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:13.314230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd416304-bb12-4158-ac85-4fe1426501fd None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:13.314525 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd416304-bb12-4158-ac85-4fe1426501fd None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:13.314790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd416304-bb12-4158-ac85-4fe1426501fd None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:13.315462 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd416304-bb12-4158-ac85-4fe1426501fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:13.315823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd416304-bb12-4158-ac85-4fe1426501fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:13.317559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd416304-bb12-4158-ac85-4fe1426501fd None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:13.329345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 193/389] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:13.335575 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd416304-bb12-4158-ac85-4fe1426501fd None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:13.354629 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:13.418607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a773b3-d811-4307-99bf-af7f5ad5284c None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:13.460079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22068a8a-fc31-499b-a7c0-0df3ad86aea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:13.460735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22068a8a-fc31-499b-a7c0-0df3ad86aea1 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:13.461013 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22068a8a-fc31-499b-a7c0-0df3ad86aea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:13.461698 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22068a8a-fc31-499b-a7c0-0df3ad86aea1 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:13.463157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22068a8a-fc31-499b-a7c0-0df3ad86aea1 None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:13.464746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22068a8a-fc31-499b-a7c0-0df3ad86aea1 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:13.466953 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 197/390] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:13 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:13.468833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4821-fc74-41cf-9d99-828d32b5fe0c None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:13.469112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4821-fc74-41cf-9d99-828d32b5fe0c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:13.469381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4821-fc74-41cf-9d99-828d32b5fe0c None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:13.471029 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 198/391] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:13.472635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd416304-bb12-4158-ac85-4fe1426501fd None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:13.475106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb4821-fc74-41cf-9d99-828d32b5fe0c None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:13.512306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f29afe-8d6e-4c94-9ffa-6ed37053367b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:13.513572 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f29afe-8d6e-4c94-9ffa-6ed37053367b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:13.514119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f29afe-8d6e-4c94-9ffa-6ed37053367b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:13.514657 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f29afe-8d6e-4c94-9ffa-6ed37053367b None admin] PATH_INFO: `/v3/groups/cfed87f511354222848ae0fb14591553` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:13.516000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f29afe-8d6e-4c94-9ffa-6ed37053367b None admin] RBAC: Authorizing `identity:get_group(group_id=cfed87f511354222848ae0fb14591553)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:13.518560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f29afe-8d6e-4c94-9ffa-6ed37053367b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:13.520666 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 194/392] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:13 2022] GET /identity/v3/groups/cfed87f511354222848ae0fb14591553 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:13.521992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3a819-c4a1-45c6-85c5-fb8b5dd819d3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:13.522570 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3a819-c4a1-45c6-85c5-fb8b5dd819d3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:13.523109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3a819-c4a1-45c6-85c5-fb8b5dd819d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:13.538977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:13.546249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-fab3a819-c4a1-45c6-85c5-fb8b5dd819d3 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:13.548352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-fab3a819-c4a1-45c6-85c5-fb8b5dd819d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:13.607970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb4821-fc74-41cf-9d99-828d32b5fe0c None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:13.620888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:13.643703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 195/393] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:13.644879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3a819-c4a1-45c6-85c5-fb8b5dd819d3 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:13.653092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68022e83-a7d0-4eb5-8fca-fff9fcb4273f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:13.653817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68022e83-a7d0-4eb5-8fca-fff9fcb4273f None admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:13.654092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68022e83-a7d0-4eb5-8fca-fff9fcb4273f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:13.654365 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68022e83-a7d0-4eb5-8fca-fff9fcb4273f None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b/groups/cfed87f511354222848ae0fb14591553/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:13.659319 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68022e83-a7d0-4eb5-8fca-fff9fcb4273f None admin] RBAC: Authorizing `identity:create_grant(project_id=c05ccc8ff85b456080a0930ce5f49c5b, group_id=cfed87f511354222848ae0fb14591553, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:13.660860 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68022e83-a7d0-4eb5-8fca-fff9fcb4273f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:13.684543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 199/394] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Sep 9 16:18:13 2022] PUT /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b/groups/cfed87f511354222848ae0fb14591553/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:18:13.782017 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3a819-c4a1-45c6-85c5-fb8b5dd819d3 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:13.824421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8dc0d-245f-43c3-b71b-ce271c1df4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:13.825080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8dc0d-245f-43c3-b71b-ce271c1df4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:13.825358 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8dc0d-245f-43c3-b71b-ce271c1df4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:13.825629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8dc0d-245f-43c3-b71b-ce271c1df4d5 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:13.826818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8dc0d-245f-43c3-b71b-ce271c1df4d5 None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:13.828335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8dc0d-245f-43c3-b71b-ce271c1df4d5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:13.830021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 196/395] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:13 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:13.836128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68022e83-a7d0-4eb5-8fca-fff9fcb4273f None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:13.982595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68022e83-a7d0-4eb5-8fca-fff9fcb4273f None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:14.010002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8dc0d-245f-43c3-b71b-ce271c1df4d5 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:14.027138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:14.029584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36374d-6bd6-4f84-8c3c-ea9d01bb5fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:14.030696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36374d-6bd6-4f84-8c3c-ea9d01bb5fba None admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:14.031174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36374d-6bd6-4f84-8c3c-ea9d01bb5fba None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:14.031652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36374d-6bd6-4f84-8c3c-ea9d01bb5fba None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b/groups/cfed87f511354222848ae0fb14591553/roles/0d40b3176f78460f9c4a29c1e9ef843e` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:14.034633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36374d-6bd6-4f84-8c3c-ea9d01bb5fba None admin] RBAC: Authorizing `identity:create_grant(project_id=c05ccc8ff85b456080a0930ce5f49c5b, group_id=cfed87f511354222848ae0fb14591553, role_id=0d40b3176f78460f9c4a29c1e9ef843e)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:14.036675 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36374d-6bd6-4f84-8c3c-ea9d01bb5fba None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:14.053890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 200/396] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Sep 9 16:18:13 2022] PUT /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b/groups/cfed87f511354222848ae0fb14591553/roles/0d40b3176f78460f9c4a29c1e9ef843e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:18:14.141502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8dc0d-245f-43c3-b71b-ce271c1df4d5 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:14.185447 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514b823-ab75-459a-8b29-b1c576c9123c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:14.186349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b823-ab75-459a-8b29-b1c576c9123c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:14.186860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b823-ab75-459a-8b29-b1c576c9123c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:14.187261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b823-ab75-459a-8b29-b1c576c9123c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:14.203718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514b823-ab75-459a-8b29-b1c576c9123c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-f514b823-ab75-459a-8b29-b1c576c9123c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:14.209455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:14.210673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:14.211760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:14.211760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:14.211760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:14.211760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:14.211760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:14.211760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:14.211760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:14.211760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:14.211760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:14.211760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:14.211760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:14.211760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:14.218415 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 197/397] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:14.219834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36374d-6bd6-4f84-8c3c-ea9d01bb5fba None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:14.282554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:14.365798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36374d-6bd6-4f84-8c3c-ea9d01bb5fba None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:14.405315 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8a79c-0ce7-4714-9bb7-2c1836601c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:14.405946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8a79c-0ce7-4714-9bb7-2c1836601c51 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:14.406225 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8a79c-0ce7-4714-9bb7-2c1836601c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:14.406511 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8a79c-0ce7-4714-9bb7-2c1836601c51 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:14.407180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8a79c-0ce7-4714-9bb7-2c1836601c51 None admin] RBAC: Adding query filter params (name=member) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:14.407482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8a79c-0ce7-4714-9bb7-2c1836601c51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:14.408858 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8a79c-0ce7-4714-9bb7-2c1836601c51 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:14.417924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 201/398] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:18:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:14.423169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514b823-ab75-459a-8b29-b1c576c9123c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:14.530615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:14.570606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514b823-ab75-459a-8b29-b1c576c9123c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:14.605536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b31bec-01f1-4c53-b517-c792fc4e83a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:14.606651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b31bec-01f1-4c53-b517-c792fc4e83a2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:14.606651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b31bec-01f1-4c53-b517-c792fc4e83a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:14.606816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b31bec-01f1-4c53-b517-c792fc4e83a2 None admin] PATH_INFO: `/v3/groups/cfed87f511354222848ae0fb14591553` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:14.607965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b31bec-01f1-4c53-b517-c792fc4e83a2 None admin] RBAC: Authorizing `identity:get_group(group_id=cfed87f511354222848ae0fb14591553)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:14.609423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b31bec-01f1-4c53-b517-c792fc4e83a2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:14.611562 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 198/399] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:14 2022] GET /identity/v3/groups/cfed87f511354222848ae0fb14591553 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:14.616987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8a79c-0ce7-4714-9bb7-2c1836601c51 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:14.766520 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:14.780136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8a79c-0ce7-4714-9bb7-2c1836601c51 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:14.835532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3f46f-4751-4caf-ab11-4fb640786ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:14.836123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3f46f-4751-4caf-ab11-4fb640786ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:14.836403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3f46f-4751-4caf-ab11-4fb640786ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:14.836673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3f46f-4751-4caf-ab11-4fb640786ea5 None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:14.837924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3f46f-4751-4caf-ab11-4fb640786ea5 None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:14.839464 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3f46f-4751-4caf-ab11-4fb640786ea5 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:14.841558 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 202/400] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:14 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:14.847557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b31bec-01f1-4c53-b517-c792fc4e83a2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:14.964223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfbb85-d419-4a84-85e1-4d6de55b8bf6 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:14.964523 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfbb85-d419-4a84-85e1-4d6de55b8bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:14.964778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfbb85-d419-4a84-85e1-4d6de55b8bf6 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:14.967396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 203/401] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:18:14.971861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da642e7f-c104-472e-ab2b-4bf979354f6a None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:14.972142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da642e7f-c104-472e-ab2b-4bf979354f6a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:14.972412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da642e7f-c104-472e-ab2b-4bf979354f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:14.982166 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b31bec-01f1-4c53-b517-c792fc4e83a2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:14.994752 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-da642e7f-c104-472e-ab2b-4bf979354f6a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:14.996273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-da642e7f-c104-472e-ab2b-4bf979354f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:15.016263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:15.021651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86840be3-4f08-4ee4-84eb-e89e5a48e6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:15.022830 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86840be3-4f08-4ee4-84eb-e89e5a48e6a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:15.023479 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86840be3-4f08-4ee4-84eb-e89e5a48e6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:15.024095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86840be3-4f08-4ee4-84eb-e89e5a48e6a2 None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401/groups/cfed87f511354222848ae0fb14591553/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:15.026010 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86840be3-4f08-4ee4-84eb-e89e5a48e6a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b1e73957f8a494d88eab174721e9401, group_id=cfed87f511354222848ae0fb14591553, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:15.027508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86840be3-4f08-4ee4-84eb-e89e5a48e6a2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:15.043751 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 199/402] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Sep 9 16:18:14 2022] PUT /identity/v3/projects/0b1e73957f8a494d88eab174721e9401/groups/cfed87f511354222848ae0fb14591553/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:18:15.045150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffac14d-2d66-41d0-879b-b10e1aa9d29b None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:15.045431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffac14d-2d66-41d0-879b-b10e1aa9d29b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:15.045762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffac14d-2d66-41d0-879b-b10e1aa9d29b None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:15.047440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 200/403] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:15.053509 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c55fa6-728e-4755-b4d8-023b7cd8a47c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:15.053925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c55fa6-728e-4755-b4d8-023b7cd8a47c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:15.054170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c55fa6-728e-4755-b4d8-023b7cd8a47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:15.077154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-97c55fa6-728e-4755-b4d8-023b7cd8a47c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:15.078436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-97c55fa6-728e-4755-b4d8-023b7cd8a47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:15.089570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 204/404] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:15.190225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 201/405] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:18:15.446659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da642e7f-c104-472e-ab2b-4bf979354f6a None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:15.462829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c55fa6-728e-4755-b4d8-023b7cd8a47c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:15.578788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:15.587811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c55fa6-728e-4755-b4d8-023b7cd8a47c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:15.603317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da642e7f-c104-472e-ab2b-4bf979354f6a None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:15.634914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:15.653717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b594d46-4289-4563-94ee-a799ca152a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:15.654359 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b594d46-4289-4563-94ee-a799ca152a74 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:15.654667 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b594d46-4289-4563-94ee-a799ca152a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:15.654966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b594d46-4289-4563-94ee-a799ca152a74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:15.667086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464fa70-6733-4b79-8ce0-e6101ff1686a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:15.667779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464fa70-6733-4b79-8ce0-e6101ff1686a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:15.668058 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464fa70-6733-4b79-8ce0-e6101ff1686a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:15.668325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464fa70-6733-4b79-8ce0-e6101ff1686a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:15.677789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b594d46-4289-4563-94ee-a799ca152a74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-5b594d46-4289-4563-94ee-a799ca152a74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:15.682810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:15.683662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:15.684423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:15.684423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:15.684423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:15.684423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:15.684423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:15.684423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:15.684423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:15.684423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:15.684423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:15.684423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:15.684423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:18:15.684423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:15.691893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 202/406] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:18:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:15.695296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464fa70-6733-4b79-8ce0-e6101ff1686a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:15.699505 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b594d46-4289-4563-94ee-a799ca152a74 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-c464fa70-6733-4b79-8ce0-e6101ff1686a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:15.700287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:15.701128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:15.701904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:15.701904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:15.701904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:15.701904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:15.701904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:15.701904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:15.701904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:15.701904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:15.701904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:15.701904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:15.701904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:18:15.701904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:15.701904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 205/407] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:18:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:15.706937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464fa70-6733-4b79-8ce0-e6101ff1686a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:15.810929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464fa70-6733-4b79-8ce0-e6101ff1686a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:15.846206 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edb884-2142-4a59-a649-a64c185a7b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:15.846850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edb884-2142-4a59-a649-a64c185a7b00 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:15.847124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edb884-2142-4a59-a649-a64c185a7b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:15.847389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edb884-2142-4a59-a649-a64c185a7b00 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:15.848044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edb884-2142-4a59-a649-a64c185a7b00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:15.848348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edb884-2142-4a59-a649-a64c185a7b00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:15.849726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edb884-2142-4a59-a649-a64c185a7b00 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:15.858436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 206/408] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:15.864374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edb884-2142-4a59-a649-a64c185a7b00 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:15.906891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:15.964770 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edb884-2142-4a59-a649-a64c185a7b00 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:15.989288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b594d46-4289-4563-94ee-a799ca152a74 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:15.999304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3369578-f5b0-4049-969c-468fb1b01d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:15.999943 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3369578-f5b0-4049-969c-468fb1b01d71 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:16.000217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3369578-f5b0-4049-969c-468fb1b01d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:16.000486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3369578-f5b0-4049-969c-468fb1b01d71 None admin] PATH_INFO: `/v3/groups/b771303ebdf949e7861d16757a60e95f` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:16.001993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3369578-f5b0-4049-969c-468fb1b01d71 None admin] RBAC: Authorizing `identity:get_group(group_id=b771303ebdf949e7861d16757a60e95f)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:16.003466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3369578-f5b0-4049-969c-468fb1b01d71 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:16.005328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 207/409] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:15 2022] GET /identity/v3/groups/b771303ebdf949e7861d16757a60e95f => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:16.011147 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3369578-f5b0-4049-969c-468fb1b01d71 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:16.094959 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:16.110122 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e275a7-9f46-4be9-84b6-e173b76a7a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:16.110777 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e275a7-9f46-4be9-84b6-e173b76a7a02 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:16.111054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e275a7-9f46-4be9-84b6-e173b76a7a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:16.111321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e275a7-9f46-4be9-84b6-e173b76a7a02 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:16.111603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3369578-f5b0-4049-969c-468fb1b01d71 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:16.111995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e275a7-9f46-4be9-84b6-e173b76a7a02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:16.112294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e275a7-9f46-4be9-84b6-e173b76a7a02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:16.113677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e275a7-9f46-4be9-84b6-e173b76a7a02 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:16.135244 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 203/410] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:16.142230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e275a7-9f46-4be9-84b6-e173b76a7a02 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:16.150971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73732df7-d76e-4440-abbd-c134fc835d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:16.151623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73732df7-d76e-4440-abbd-c134fc835d93 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:16.151905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73732df7-d76e-4440-abbd-c134fc835d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:16.152178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73732df7-d76e-4440-abbd-c134fc835d93 None admin] PATH_INFO: `/v3/projects/f5e628b418b44cb0b8fcf5460ded8ac8` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:16.153305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73732df7-d76e-4440-abbd-c134fc835d93 None admin] RBAC: Authorizing `identity:get_project(project_id=f5e628b418b44cb0b8fcf5460ded8ac8)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:16.154788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73732df7-d76e-4440-abbd-c134fc835d93 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:16.156699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 208/411] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:16 2022] GET /identity/v3/projects/f5e628b418b44cb0b8fcf5460ded8ac8 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:16.162190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73732df7-d76e-4440-abbd-c134fc835d93 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:16.258099 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:16.262240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73732df7-d76e-4440-abbd-c134fc835d93 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:16.296134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e60314-4556-4536-8b6a-da9c119f29f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:16.296814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e60314-4556-4536-8b6a-da9c119f29f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:16.297096 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e60314-4556-4536-8b6a-da9c119f29f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:16.297373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e60314-4556-4536-8b6a-da9c119f29f7 None admin] PATH_INFO: `/v3/projects/f5e628b418b44cb0b8fcf5460ded8ac8/groups/b771303ebdf949e7861d16757a60e95f/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:16.299583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e60314-4556-4536-8b6a-da9c119f29f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5e628b418b44cb0b8fcf5460ded8ac8, group_id=b771303ebdf949e7861d16757a60e95f, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:16.301049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e60314-4556-4536-8b6a-da9c119f29f7 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:16.322812 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 209/412] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Sep 9 16:18:16 2022] PUT /identity/v3/projects/f5e628b418b44cb0b8fcf5460ded8ac8/groups/b771303ebdf949e7861d16757a60e95f/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:18:16.399395 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e275a7-9f46-4be9-84b6-e173b76a7a02 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:16.445526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbf3bd-e525-4577-8743-b702cd1bad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:16.446183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbf3bd-e525-4577-8743-b702cd1bad0b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:16.446458 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbf3bd-e525-4577-8743-b702cd1bad0b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:16.446756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbf3bd-e525-4577-8743-b702cd1bad0b None admin] PATH_INFO: `/v3/groups/cfed87f511354222848ae0fb14591553` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:16.448038 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbf3bd-e525-4577-8743-b702cd1bad0b None admin] RBAC: Authorizing `identity:get_group(group_id=cfed87f511354222848ae0fb14591553)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:16.449508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbf3bd-e525-4577-8743-b702cd1bad0b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:16.451537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 204/413] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:16 2022] GET /identity/v3/groups/cfed87f511354222848ae0fb14591553 => generated 226 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:16.456931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e60314-4556-4536-8b6a-da9c119f29f7 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:16.470902 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:16.506181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3af0f7-89fd-4b2a-879e-d5b18b3c2b4a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:16.506453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3af0f7-89fd-4b2a-879e-d5b18b3c2b4a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:16.506804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3af0f7-89fd-4b2a-879e-d5b18b3c2b4a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:16.508541 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 205/414] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:16.518022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a81548-5253-4564-bdc1-288984510e80 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:16.518331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a81548-5253-4564-bdc1-288984510e80 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:16.518636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a81548-5253-4564-bdc1-288984510e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:16.559565 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-11a81548-5253-4564-bdc1-288984510e80 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:16.560852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-11a81548-5253-4564-bdc1-288984510e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:16.587482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e60314-4556-4536-8b6a-da9c119f29f7 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:16.635219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f89d00-37b2-4087-ad78-ed269ccae9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:16.635873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f89d00-37b2-4087-ad78-ed269ccae9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:16.636148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f89d00-37b2-4087-ad78-ed269ccae9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:16.636419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f89d00-37b2-4087-ad78-ed269ccae9f4 None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:16.637684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f89d00-37b2-4087-ad78-ed269ccae9f4 None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:16.643886 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f89d00-37b2-4087-ad78-ed269ccae9f4 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:16.646103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 210/415] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:16 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:16.652813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f89d00-37b2-4087-ad78-ed269ccae9f4 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:16.697463 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 206/416] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:18:16.759023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:16.788286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f89d00-37b2-4087-ad78-ed269ccae9f4 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:16.829784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924249ae-69ec-44c4-94d7-7625490674f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:16.830457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924249ae-69ec-44c4-94d7-7625490674f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:16.830760 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924249ae-69ec-44c4-94d7-7625490674f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:16.831078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924249ae-69ec-44c4-94d7-7625490674f8 None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401/groups/cfed87f511354222848ae0fb14591553/roles/0d40b3176f78460f9c4a29c1e9ef843e` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:16.833211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924249ae-69ec-44c4-94d7-7625490674f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b1e73957f8a494d88eab174721e9401, group_id=cfed87f511354222848ae0fb14591553, role_id=0d40b3176f78460f9c4a29c1e9ef843e)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:16.834687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924249ae-69ec-44c4-94d7-7625490674f8 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:16.851325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 211/417] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Sep 9 16:18:16 2022] PUT /identity/v3/projects/0b1e73957f8a494d88eab174721e9401/groups/cfed87f511354222848ae0fb14591553/roles/0d40b3176f78460f9c4a29c1e9ef843e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:18:16.856234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2066fc-aeb2-4746-9c60-04e78e790b5e None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:16.856529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2066fc-aeb2-4746-9c60-04e78e790b5e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:16.856824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2066fc-aeb2-4746-9c60-04e78e790b5e None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:16.858695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 207/418] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:16.868437 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94b1ec-3d5e-499e-b842-11da20c1247a None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:16.868722 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94b1ec-3d5e-499e-b842-11da20c1247a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:16.869086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94b1ec-3d5e-499e-b842-11da20c1247a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:16.889518 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-3b94b1ec-3d5e-499e-b842-11da20c1247a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:16.890629 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-3b94b1ec-3d5e-499e-b842-11da20c1247a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:16.979607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 212/419] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:17.011587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2066fc-aeb2-4746-9c60-04e78e790b5e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:17.064656 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:17.124819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2066fc-aeb2-4746-9c60-04e78e790b5e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:17.160397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be5f00-2cbf-467d-9e46-f433db8fef29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:17.161047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be5f00-2cbf-467d-9e46-f433db8fef29 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:17.161319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be5f00-2cbf-467d-9e46-f433db8fef29 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:17.161581 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be5f00-2cbf-467d-9e46-f433db8fef29 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:17.172901 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be5f00-2cbf-467d-9e46-f433db8fef29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-27be5f00-2cbf-467d-9e46-f433db8fef29 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:17.177769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:17.178800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:17.179587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:17.179587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:17.179587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:17.179587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:17.179587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:17.179587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:17.179587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:17.179587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:17.179587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:17.179587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:17.179587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:17.179587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:17.179587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 208/420] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:17.188581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94b1ec-3d5e-499e-b842-11da20c1247a None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:17.262039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:17.282900 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be5f00-2cbf-467d-9e46-f433db8fef29 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:17.304767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94b1ec-3d5e-499e-b842-11da20c1247a None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:17.341272 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdc0b7-538d-487d-8031-e172c878a86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:17.342337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdc0b7-538d-487d-8031-e172c878a86b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:17.342827 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdc0b7-538d-487d-8031-e172c878a86b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:17.343315 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdc0b7-538d-487d-8031-e172c878a86b None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:17.344165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdc0b7-538d-487d-8031-e172c878a86b None admin] RBAC: Adding query filter params (name=member) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:17.344667 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdc0b7-538d-487d-8031-e172c878a86b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:17.346230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdc0b7-538d-487d-8031-e172c878a86b None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:17.355395 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 213/421] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:18:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:17.361436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdc0b7-538d-487d-8031-e172c878a86b None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:17.458388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be5f00-2cbf-467d-9e46-f433db8fef29 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:17.462151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:17.529674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb6c59-f142-4595-84ca-f704b9b92b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:17.530666 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb6c59-f142-4595-84ca-f704b9b92b97 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:17.531150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb6c59-f142-4595-84ca-f704b9b92b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:17.531618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb6c59-f142-4595-84ca-f704b9b92b97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:17.544045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb6c59-f142-4595-84ca-f704b9b92b97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c2bb6c59-f142-4595-84ca-f704b9b92b97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:17.549110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:17.549990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:17.550795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:17.550795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:17.550795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:17.550795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:17.550795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:17.550795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:17.550795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:17.550795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:17.550795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:17.550795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:17.550795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:18:17.550795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:17.553445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 209/422] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:18:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:17.558358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdc0b7-538d-487d-8031-e172c878a86b None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:17.559789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb6c59-f142-4595-84ca-f704b9b92b97 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:17.606963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:17.625340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fdb22-a659-4adc-b75a-077fc1c34949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:17.626264 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fdb22-a659-4adc-b75a-077fc1c34949 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:17.626775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fdb22-a659-4adc-b75a-077fc1c34949 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:17.627249 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fdb22-a659-4adc-b75a-077fc1c34949 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:17.732775 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb6c59-f142-4595-84ca-f704b9b92b97 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:17.744708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fdb22-a659-4adc-b75a-077fc1c34949 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:17.746228 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fdb22-a659-4adc-b75a-077fc1c34949 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:17.797301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c7429-2478-4fff-a166-f47a5342eef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:17.797827 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c7429-2478-4fff-a166-f47a5342eef0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:17.798100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c7429-2478-4fff-a166-f47a5342eef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:17.798365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c7429-2478-4fff-a166-f47a5342eef0 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:17.799052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c7429-2478-4fff-a166-f47a5342eef0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:17.799347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c7429-2478-4fff-a166-f47a5342eef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:17.800732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c7429-2478-4fff-a166-f47a5342eef0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:17.810031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 210/423] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:17.815808 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c7429-2478-4fff-a166-f47a5342eef0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:17.864849 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:17.939906 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 214/424] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 16:18:17.947511 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fdb22-a659-4adc-b75a-077fc1c34949 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:17.976831 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c7429-2478-4fff-a166-f47a5342eef0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:18.002070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:18.015998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437df67-e154-40e8-b265-59bb4943e98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:18.016975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437df67-e154-40e8-b265-59bb4943e98d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:18.017497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437df67-e154-40e8-b265-59bb4943e98d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:18.017950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437df67-e154-40e8-b265-59bb4943e98d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:18.084417 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fdb22-a659-4adc-b75a-077fc1c34949 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:18.095603 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a437df67-e154-40e8-b265-59bb4943e98d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:18.096929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a437df67-e154-40e8-b265-59bb4943e98d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:18.126689 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:18.129186 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af03df-2493-422e-8701-29a2683d7950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:18.130069 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af03df-2493-422e-8701-29a2683d7950 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:18.130555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af03df-2493-422e-8701-29a2683d7950 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:18.131064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af03df-2493-422e-8701-29a2683d7950 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:18.132426 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af03df-2493-422e-8701-29a2683d7950 None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:18.134078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af03df-2493-422e-8701-29a2683d7950 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:18.136245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 215/425] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:17 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:18.137632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8dfe6-f448-4d58-a49c-bb7e42b847d5 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:18.138122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8dfe6-f448-4d58-a49c-bb7e42b847d5 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:18.138605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8dfe6-f448-4d58-a49c-bb7e42b847d5 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:18.139994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 216/426] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:18.144968 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8dfe6-f448-4d58-a49c-bb7e42b847d5 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:18.221591 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 211/427] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 16:18:18.222906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cbaeb-6751-40e0-ab0b-ac95ef5bc070 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:18.223185 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cbaeb-6751-40e0-ab0b-ac95ef5bc070 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:18.223452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cbaeb-6751-40e0-ab0b-ac95ef5bc070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:18.245341 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:18.247956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4a1cbaeb-6751-40e0-ab0b-ac95ef5bc070 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:18.249313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4a1cbaeb-6751-40e0-ab0b-ac95ef5bc070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:18.254745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8dfe6-f448-4d58-a49c-bb7e42b847d5 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:18.290240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5fccc-a781-45a8-ab53-1c50c2ef28c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:18.291294 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5fccc-a781-45a8-ab53-1c50c2ef28c6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:18.291762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5fccc-a781-45a8-ab53-1c50c2ef28c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:18.292247 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5fccc-a781-45a8-ab53-1c50c2ef28c6 None admin] PATH_INFO: `/v3/groups/cfed87f511354222848ae0fb14591553` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:18.293596 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5fccc-a781-45a8-ab53-1c50c2ef28c6 None admin] RBAC: Authorizing `identity:get_group(group_id=cfed87f511354222848ae0fb14591553)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:18.295504 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5fccc-a781-45a8-ab53-1c50c2ef28c6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:18.297652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 217/428] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:18 2022] GET /identity/v3/groups/cfed87f511354222848ae0fb14591553 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:18.298678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5fccc-a781-45a8-ab53-1c50c2ef28c6 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:18.334610 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 212/429] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:18.334610 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cbaeb-6751-40e0-ab0b-ac95ef5bc070 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:18.405560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:18.455190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5fccc-a781-45a8-ab53-1c50c2ef28c6 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:18.488699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cbaeb-6751-40e0-ab0b-ac95ef5bc070 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:18.492195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b3fc2-7ae1-4161-881c-eb0810218084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:18.492829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b3fc2-7ae1-4161-881c-eb0810218084 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:18.493102 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b3fc2-7ae1-4161-881c-eb0810218084 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:18.493367 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b3fc2-7ae1-4161-881c-eb0810218084 None admin] PATH_INFO: `/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:18.494509 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b3fc2-7ae1-4161-881c-eb0810218084 None admin] RBAC: Authorizing `identity:get_project(project_id=c05ccc8ff85b456080a0930ce5f49c5b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:18.496758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b3fc2-7ae1-4161-881c-eb0810218084 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:18.498715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 218/430] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:18 2022] GET /identity/v3/projects/c05ccc8ff85b456080a0930ce5f49c5b => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:18.505303 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b3fc2-7ae1-4161-881c-eb0810218084 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:18.556918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdec547-ad92-4abe-96fe-06fedd721c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:18.557530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdec547-ad92-4abe-96fe-06fedd721c6c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:18.557801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdec547-ad92-4abe-96fe-06fedd721c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:18.558071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdec547-ad92-4abe-96fe-06fedd721c6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:18.558823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdec547-ad92-4abe-96fe-06fedd721c6c None admin] RBAC: Adding query filter params (group.id=cfed87f511354222848ae0fb14591553, role.id=9e02fa627e5c4388b0aa7182f1707895, scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:18.559134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdec547-ad92-4abe-96fe-06fedd721c6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:18.560578 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdec547-ad92-4abe-96fe-06fedd721c6c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:18.570231 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:18.579784 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 213/431] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Sep 9 16:18:18 2022] GET /identity/v3/role_assignments?group.id=cfed87f511354222848ae0fb14591553&scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b&role.id=9e02fa627e5c4388b0aa7182f1707895 => generated 611 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:18.641397 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b3fc2-7ae1-4161-881c-eb0810218084 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:18.646397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdec547-ad92-4abe-96fe-06fedd721c6c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:18.685465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d6073-4794-48f5-85c1-b70e3ac68bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:18.687086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d6073-4794-48f5-85c1-b70e3ac68bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:18.687394 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d6073-4794-48f5-85c1-b70e3ac68bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:18.687665 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d6073-4794-48f5-85c1-b70e3ac68bc2 None admin] PATH_INFO: `/v3/groups/cfed87f511354222848ae0fb14591553` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:18.688863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d6073-4794-48f5-85c1-b70e3ac68bc2 None admin] RBAC: Authorizing `identity:get_group(group_id=cfed87f511354222848ae0fb14591553)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:18.690323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d6073-4794-48f5-85c1-b70e3ac68bc2 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:18.692613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 219/432] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:18 2022] GET /identity/v3/groups/cfed87f511354222848ae0fb14591553 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:18.698298 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d6073-4794-48f5-85c1-b70e3ac68bc2 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:18.802758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdec547-ad92-4abe-96fe-06fedd721c6c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:18.840379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:18.870531 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b223b-f4c3-445d-9181-0a669782bd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:18.871204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b223b-f4c3-445d-9181-0a669782bd67 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:18.871480 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b223b-f4c3-445d-9181-0a669782bd67 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:18.871751 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b223b-f4c3-445d-9181-0a669782bd67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:18.886526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b223b-f4c3-445d-9181-0a669782bd67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:18.890000 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:18.894181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d6073-4794-48f5-85c1-b70e3ac68bc2 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-eb0b223b-f4c3-445d-9181-0a669782bd67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:18.894643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:18.895484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:18.896932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:18.896932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:18.896932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:18.896932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:18.896932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:18.896932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:18.896932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:18.896932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:18.896932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:18.896932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:18.896932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 16:18:18.896932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:18.896932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 214/433] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:18.898653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0473736-108a-43b1-8f6d-f10c71417598 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:18.898834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0473736-108a-43b1-8f6d-f10c71417598 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:18.899100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0473736-108a-43b1-8f6d-f10c71417598 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:18.901092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 215/434] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:18.903595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0473736-108a-43b1-8f6d-f10c71417598 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:18.946133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b951a-81c4-479d-ae87-aea092a0a555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:18.947094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b951a-81c4-479d-ae87-aea092a0a555 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:18.947568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b951a-81c4-479d-ae87-aea092a0a555 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:18.948024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b951a-81c4-479d-ae87-aea092a0a555 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:18.948925 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b951a-81c4-479d-ae87-aea092a0a555 None admin] RBAC: Adding query filter params (group.id=cfed87f511354222848ae0fb14591553, role.id=0d40b3176f78460f9c4a29c1e9ef843e, scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:18.949408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b951a-81c4-479d-ae87-aea092a0a555 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:18.951144 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b951a-81c4-479d-ae87-aea092a0a555 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:18.959553 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 220/435] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Sep 9 16:18:18 2022] GET /identity/v3/role_assignments?group.id=cfed87f511354222848ae0fb14591553&scope.project.id=c05ccc8ff85b456080a0930ce5f49c5b&role.id=0d40b3176f78460f9c4a29c1e9ef843e => generated 611 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:18.961230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c503949-ed8f-4c0d-9867-8d8f178f253c None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:18.961710 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c503949-ed8f-4c0d-9867-8d8f178f253c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:18.962152 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c503949-ed8f-4c0d-9867-8d8f178f253c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:18.985197 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-5c503949-ed8f-4c0d-9867-8d8f178f253c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:18.986882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5c503949-ed8f-4c0d-9867-8d8f178f253c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:19.043949 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0473736-108a-43b1-8f6d-f10c71417598 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:19.084857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 221/436] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:19.100929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501ce86-7dab-46fa-893f-cd6dfdb1d2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:19.101802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501ce86-7dab-46fa-893f-cd6dfdb1d2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:19.102290 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501ce86-7dab-46fa-893f-cd6dfdb1d2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:19.102736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501ce86-7dab-46fa-893f-cd6dfdb1d2b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:19.103620 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501ce86-7dab-46fa-893f-cd6dfdb1d2b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:19.104073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501ce86-7dab-46fa-893f-cd6dfdb1d2b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:19.105726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501ce86-7dab-46fa-893f-cd6dfdb1d2b2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:19.117107 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 216/437] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:18:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:19.126792 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c503949-ed8f-4c0d-9867-8d8f178f253c None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:19.227942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:19.237391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c503949-ed8f-4c0d-9867-8d8f178f253c None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:19.274158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0bbad-188d-47ce-a90b-2565d4ac4402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:19.274815 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0bbad-188d-47ce-a90b-2565d4ac4402 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:19.275148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0bbad-188d-47ce-a90b-2565d4ac4402 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:19.275421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0bbad-188d-47ce-a90b-2565d4ac4402 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:19.349828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0bbad-188d-47ce-a90b-2565d4ac4402 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:19.351416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0bbad-188d-47ce-a90b-2565d4ac4402 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:19.374212 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501ce86-7dab-46fa-893f-cd6dfdb1d2b2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:19.430592 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:19.463987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:19.474603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 222/438] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:19.475941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b58578-5be7-47f6-95ad-47f6987a8609 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:19.476665 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b58578-5be7-47f6-95ad-47f6987a8609 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:19.477158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b58578-5be7-47f6-95ad-47f6987a8609 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:19.479165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 223/439] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:19.480277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b58578-5be7-47f6-95ad-47f6987a8609 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:19.504816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501ce86-7dab-46fa-893f-cd6dfdb1d2b2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:19.545448 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955dba28-a968-458b-88b9-c5982c5542d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:19.546084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dba28-a968-458b-88b9-c5982c5542d4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:19.546364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dba28-a968-458b-88b9-c5982c5542d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:19.546681 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dba28-a968-458b-88b9-c5982c5542d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:19.558481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955dba28-a968-458b-88b9-c5982c5542d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-955dba28-a968-458b-88b9-c5982c5542d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:19.563913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:19.564781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:19.569928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:19.569928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:19.569928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:19.569928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:19.569928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:19.569928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:19.569928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:19.569928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:19.569928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:19.569928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:19.569928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:18:19.569928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:19.569928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 217/440] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:18:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:19.569928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa39e73-946a-4f7f-98b1-836e3417ce5f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:19.570647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa39e73-946a-4f7f-98b1-836e3417ce5f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:19.570647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa39e73-946a-4f7f-98b1-836e3417ce5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:19.588118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-faa39e73-946a-4f7f-98b1-836e3417ce5f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:19.589324 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-faa39e73-946a-4f7f-98b1-836e3417ce5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:19.605374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b58578-5be7-47f6-95ad-47f6987a8609 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:19.638368 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4d4c0-1f4e-4834-845a-ceb790e8baa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:19.639280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4d4c0-1f4e-4834-845a-ceb790e8baa6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:19.639556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4d4c0-1f4e-4834-845a-ceb790e8baa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:19.639812 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4d4c0-1f4e-4834-845a-ceb790e8baa6 None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:19.640948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad4d4c0-1f4e-4834-845a-ceb790e8baa6 None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:19.642844 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad4d4c0-1f4e-4834-845a-ceb790e8baa6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:19.649060 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 224/441] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:19 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:19.649060 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4d4c0-1f4e-4834-845a-ceb790e8baa6 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:19.659319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 218/442] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:19.660011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa39e73-946a-4f7f-98b1-836e3417ce5f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:19.689969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:19.706673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:19.748838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4d4c0-1f4e-4834-845a-ceb790e8baa6 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:19.763360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa39e73-946a-4f7f-98b1-836e3417ce5f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:19.786238 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525efcc-8177-44e6-8980-8b54238720db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:19.787083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525efcc-8177-44e6-8980-8b54238720db None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:19.787356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525efcc-8177-44e6-8980-8b54238720db None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:19.787630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525efcc-8177-44e6-8980-8b54238720db None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:19.788352 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525efcc-8177-44e6-8980-8b54238720db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:19.788651 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525efcc-8177-44e6-8980-8b54238720db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:19.790052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525efcc-8177-44e6-8980-8b54238720db None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:19.796917 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c7af2-679d-40a7-a2e1-4c8fb88ce108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:19.797544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c7af2-679d-40a7-a2e1-4c8fb88ce108 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:19.797816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c7af2-679d-40a7-a2e1-4c8fb88ce108 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:19.798081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c7af2-679d-40a7-a2e1-4c8fb88ce108 None admin] PATH_INFO: `/v3/groups/cfed87f511354222848ae0fb14591553` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:19.798583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 225/443] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:19.799218 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c7af2-679d-40a7-a2e1-4c8fb88ce108 None admin] RBAC: Authorizing `identity:get_group(group_id=cfed87f511354222848ae0fb14591553)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:19.800670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c7af2-679d-40a7-a2e1-4c8fb88ce108 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:19.807014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 219/444] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:19 2022] GET /identity/v3/groups/cfed87f511354222848ae0fb14591553 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:19.807164 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525efcc-8177-44e6-8980-8b54238720db None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:19.807808 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c7af2-679d-40a7-a2e1-4c8fb88ce108 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:19.904033 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525efcc-8177-44e6-8980-8b54238720db None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:19.907606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:19.912338 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c7af2-679d-40a7-a2e1-4c8fb88ce108 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:19.927086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:19.938873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52d80a-a9e9-4351-9c6a-a2d929dde6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:19.939489 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52d80a-a9e9-4351-9c6a-a2d929dde6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:19.939764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52d80a-a9e9-4351-9c6a-a2d929dde6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:19.940025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52d80a-a9e9-4351-9c6a-a2d929dde6a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:19.946193 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e230d-d72c-4877-b4ee-38e5206bc4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:19.946821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e230d-d72c-4877-b4ee-38e5206bc4cd None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:19.947097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e230d-d72c-4877-b4ee-38e5206bc4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:19.947364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e230d-d72c-4877-b4ee-38e5206bc4cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:19.948081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e230d-d72c-4877-b4ee-38e5206bc4cd None admin] RBAC: Adding query filter params (group.id=cfed87f511354222848ae0fb14591553, role.id=9e02fa627e5c4388b0aa7182f1707895, scope.project.id=0b1e73957f8a494d88eab174721e9401) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:19.948385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e230d-d72c-4877-b4ee-38e5206bc4cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:19.949817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e230d-d72c-4877-b4ee-38e5206bc4cd None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:19.955481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 220/445] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Sep 9 16:18:19 2022] GET /identity/v3/role_assignments?group.id=cfed87f511354222848ae0fb14591553&scope.project.id=0b1e73957f8a494d88eab174721e9401&role.id=9e02fa627e5c4388b0aa7182f1707895 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:19.956245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e230d-d72c-4877-b4ee-38e5206bc4cd None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:20.028896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52d80a-a9e9-4351-9c6a-a2d929dde6a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:20.030420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52d80a-a9e9-4351-9c6a-a2d929dde6a3 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:20.068811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e230d-d72c-4877-b4ee-38e5206bc4cd None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:20.113388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f7256-e93b-425d-afad-80fe0dcf00c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:20.114034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f7256-e93b-425d-afad-80fe0dcf00c1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:20.114303 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f7256-e93b-425d-afad-80fe0dcf00c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:20.114598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f7256-e93b-425d-afad-80fe0dcf00c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:20.126675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294f7256-e93b-425d-afad-80fe0dcf00c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-294f7256-e93b-425d-afad-80fe0dcf00c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:20.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:20.134073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:20.135176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:20.135176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:20.135176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:20.135176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:20.135176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:20.135176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:20.135176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:20.135176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:20.135176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:20.135176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:20.135176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:18:20.135176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:20.135176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 221/446] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:18:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:20.139415 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f7256-e93b-425d-afad-80fe0dcf00c1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:20.148898 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 226/447] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 16:18:20.152028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52d80a-a9e9-4351-9c6a-a2d929dde6a3 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:20.167846 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:20.177924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:20.251706 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f7256-e93b-425d-afad-80fe0dcf00c1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:20.263384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52d80a-a9e9-4351-9c6a-a2d929dde6a3 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:20.292679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f45cc9-3823-4fbd-b82f-2c1ec41749ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:20.293220 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f45cc9-3823-4fbd-b82f-2c1ec41749ec None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:20.293498 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f45cc9-3823-4fbd-b82f-2c1ec41749ec None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:20.293767 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f45cc9-3823-4fbd-b82f-2c1ec41749ec None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:20.294429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f45cc9-3823-4fbd-b82f-2c1ec41749ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:20.294760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f45cc9-3823-4fbd-b82f-2c1ec41749ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:20.296159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f45cc9-3823-4fbd-b82f-2c1ec41749ec None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:20.300053 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4aa7eb-2ca6-4cff-b764-589617531919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:20.300867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4aa7eb-2ca6-4cff-b764-589617531919 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:20.301148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4aa7eb-2ca6-4cff-b764-589617531919 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:20.301416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4aa7eb-2ca6-4cff-b764-589617531919 None admin] PATH_INFO: `/v3/projects/f5e628b418b44cb0b8fcf5460ded8ac8` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:20.302563 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4aa7eb-2ca6-4cff-b764-589617531919 None admin] RBAC: Authorizing `identity:get_project(project_id=f5e628b418b44cb0b8fcf5460ded8ac8)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:20.304066 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4aa7eb-2ca6-4cff-b764-589617531919 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:20.310083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 227/448] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:20 2022] GET /identity/v3/projects/f5e628b418b44cb0b8fcf5460ded8ac8 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:20.310230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 222/449] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:20.311168 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4aa7eb-2ca6-4cff-b764-589617531919 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:20.311290 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f45cc9-3823-4fbd-b82f-2c1ec41749ec None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:20.401527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:20.402750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:20.412495 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4aa7eb-2ca6-4cff-b764-589617531919 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:20.417315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f45cc9-3823-4fbd-b82f-2c1ec41749ec None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:20.447270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2144ca6-f659-4542-9b2f-61c3068e8b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:20.447901 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2144ca6-f659-4542-9b2f-61c3068e8b8e None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:20.448180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2144ca6-f659-4542-9b2f-61c3068e8b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:20.448449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2144ca6-f659-4542-9b2f-61c3068e8b8e None admin] PATH_INFO: `/v3/groups/b771303ebdf949e7861d16757a60e95f` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:20.449579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2144ca6-f659-4542-9b2f-61c3068e8b8e None admin] RBAC: Authorizing `identity:get_group(group_id=b771303ebdf949e7861d16757a60e95f)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:20.450532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f189533-afe1-427c-a7df-05d4b6c52516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:20.451057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2144ca6-f659-4542-9b2f-61c3068e8b8e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:20.451149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f189533-afe1-427c-a7df-05d4b6c52516 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:20.451389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f189533-afe1-427c-a7df-05d4b6c52516 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:20.451657 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f189533-afe1-427c-a7df-05d4b6c52516 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:20.452627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 228/450] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:20 2022] GET /identity/v3/groups/b771303ebdf949e7861d16757a60e95f => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:20.458488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2144ca6-f659-4542-9b2f-61c3068e8b8e None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:20.517057 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f189533-afe1-427c-a7df-05d4b6c52516 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:20.518543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f189533-afe1-427c-a7df-05d4b6c52516 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:20.554899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2144ca6-f659-4542-9b2f-61c3068e8b8e None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:20.588443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfeb76d-5a94-4f14-b0a2-cc00f586953d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:20.589049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeb76d-5a94-4f14-b0a2-cc00f586953d None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:20.589326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeb76d-5a94-4f14-b0a2-cc00f586953d None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:20.589592 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeb76d-5a94-4f14-b0a2-cc00f586953d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:20.590331 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfeb76d-5a94-4f14-b0a2-cc00f586953d None admin] RBAC: Adding query filter params (group.id=b771303ebdf949e7861d16757a60e95f, role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=f5e628b418b44cb0b8fcf5460ded8ac8) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:20.590690 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfeb76d-5a94-4f14-b0a2-cc00f586953d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:20.592114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfeb76d-5a94-4f14-b0a2-cc00f586953d None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:20.597793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 229/451] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Sep 9 16:18:20 2022] GET /identity/v3/role_assignments?group.id=b771303ebdf949e7861d16757a60e95f&scope.project.id=f5e628b418b44cb0b8fcf5460ded8ac8&role.id=c6da7ba2be504943af29338ee8855f05 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:20.598879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:20.616448 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 223/452] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:20.622529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfeb76d-5a94-4f14-b0a2-cc00f586953d None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:20.649187 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:20.733500 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfeb76d-5a94-4f14-b0a2-cc00f586953d None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:20.770012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c6e43-079d-47d1-b813-66889a2bf5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:20.770681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c6e43-079d-47d1-b813-66889a2bf5df None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:20.770955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c6e43-079d-47d1-b813-66889a2bf5df None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:20.771202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c6e43-079d-47d1-b813-66889a2bf5df None admin] PATH_INFO: `/v3/projects/0b1e73957f8a494d88eab174721e9401` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:20.772388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c6e43-079d-47d1-b813-66889a2bf5df None admin] RBAC: Authorizing `identity:get_project(project_id=0b1e73957f8a494d88eab174721e9401)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:20.773820 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c6e43-079d-47d1-b813-66889a2bf5df None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:20.775994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 230/453] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:18:20 2022] GET /identity/v3/projects/0b1e73957f8a494d88eab174721e9401 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:20.782081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f189533-afe1-427c-a7df-05d4b6c52516 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:20.886359 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f189533-afe1-427c-a7df-05d4b6c52516 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:20.921433 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0471c-ef98-49cc-87bc-ed3e971f7f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:20.922051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0471c-ef98-49cc-87bc-ed3e971f7f76 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:20.922324 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0471c-ef98-49cc-87bc-ed3e971f7f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:20.922609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0471c-ef98-49cc-87bc-ed3e971f7f76 None admin] PATH_INFO: `/v3/groups/cfed87f511354222848ae0fb14591553` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:20.923829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0471c-ef98-49cc-87bc-ed3e971f7f76 None admin] RBAC: Authorizing `identity:get_group(group_id=cfed87f511354222848ae0fb14591553)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:20.925287 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0471c-ef98-49cc-87bc-ed3e971f7f76 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:20.926878 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 224/454] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Sep 9 16:18:20 2022] GET /identity/v3/groups/cfed87f511354222848ae0fb14591553 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:20.936042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c6e43-079d-47d1-b813-66889a2bf5df None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:21.028840 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:21.046185 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c6e43-079d-47d1-b813-66889a2bf5df None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:21.082466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0ebde-ff56-41e0-80e0-79c7ae14eb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:21.083099 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0ebde-ff56-41e0-80e0-79c7ae14eb91 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:21.083372 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0ebde-ff56-41e0-80e0-79c7ae14eb91 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:21.084192 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0ebde-ff56-41e0-80e0-79c7ae14eb91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:21.084351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0ebde-ff56-41e0-80e0-79c7ae14eb91 None admin] RBAC: Adding query filter params (group.id=cfed87f511354222848ae0fb14591553, role.id=0d40b3176f78460f9c4a29c1e9ef843e, scope.project.id=0b1e73957f8a494d88eab174721e9401) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:21.084765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0ebde-ff56-41e0-80e0-79c7ae14eb91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:21.086190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0ebde-ff56-41e0-80e0-79c7ae14eb91 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:21.092384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 231/455] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Sep 9 16:18:20 2022] GET /identity/v3/role_assignments?group.id=cfed87f511354222848ae0fb14591553&scope.project.id=0b1e73957f8a494d88eab174721e9401&role.id=0d40b3176f78460f9c4a29c1e9ef843e => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:23.851249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b812ff-410c-4eaf-9c6a-7b5b25a33e56 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:23.851778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b812ff-410c-4eaf-9c6a-7b5b25a33e56 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:23.851778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b812ff-410c-4eaf-9c6a-7b5b25a33e56 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:23.853388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 225/456] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:23.858477 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe1279-d3a6-468a-be75-cd9fc4b7da34 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:23.859313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe1279-d3a6-468a-be75-cd9fc4b7da34 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:23.859774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe1279-d3a6-468a-be75-cd9fc4b7da34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:23.883448 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-ecfe1279-d3a6-468a-be75-cd9fc4b7da34 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:23.884954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ecfe1279-d3a6-468a-be75-cd9fc4b7da34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:23.972588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 232/457] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:24.112484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfecc4d-8fc0-4ae0-9064-58d8e045ba2c None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:24.114305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfecc4d-8fc0-4ae0-9064-58d8e045ba2c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:24.114305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfecc4d-8fc0-4ae0-9064-58d8e045ba2c None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:24.114765 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 226/458] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:24.120419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91322816-9f2f-4ca4-9b5e-25631baf58e3 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:24.120886 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91322816-9f2f-4ca4-9b5e-25631baf58e3 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:24.121299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91322816-9f2f-4ca4-9b5e-25631baf58e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:24.151639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-91322816-9f2f-4ca4-9b5e-25631baf58e3 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:24.153153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-91322816-9f2f-4ca4-9b5e-25631baf58e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:24.249142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 233/459] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:18:24.310377 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfecc4d-8fc0-4ae0-9064-58d8e045ba2c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:24.311880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f068c-8a3e-4742-9eb2-5f78641543da None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:24.312167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f068c-8a3e-4742-9eb2-5f78641543da None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:24.312440 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f068c-8a3e-4742-9eb2-5f78641543da None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:24.314297 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 234/460] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:18:24.326229 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e83485-33e0-4d07-98b5-6e8100ae19be None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:24.326519 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e83485-33e0-4d07-98b5-6e8100ae19be None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:24.326803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e83485-33e0-4d07-98b5-6e8100ae19be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:24.368034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-54e83485-33e0-4d07-98b5-6e8100ae19be None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:24.369153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-54e83485-33e0-4d07-98b5-6e8100ae19be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:24.390875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:24.402856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:24.451335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfecc4d-8fc0-4ae0-9064-58d8e045ba2c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:24.485688 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4928b4-1653-4b09-890b-944750848158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:24.486323 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4928b4-1653-4b09-890b-944750848158 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:24.486630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4928b4-1653-4b09-890b-944750848158 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:24.486898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4928b4-1653-4b09-890b-944750848158 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:24.497898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4928b4-1653-4b09-890b-944750848158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:24.499707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4928b4-1653-4b09-890b-944750848158 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-be4928b4-1653-4b09-890b-944750848158 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:24.515252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:24.516423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:24.517483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:24.517483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:24.517483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:24.519164 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 227/461] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:24.531492 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4928b4-1653-4b09-890b-944750848158 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:24.532287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 235/462] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:24.574162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e83485-33e0-4d07-98b5-6e8100ae19be None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:24.708019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e83485-33e0-4d07-98b5-6e8100ae19be None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:24.714152 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4928b4-1653-4b09-890b-944750848158 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:24.716979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:24.745093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a7b5-7217-4431-9f19-a60c6db4e7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:24.746162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a7b5-7217-4431-9f19-a60c6db4e7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:24.746740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a7b5-7217-4431-9f19-a60c6db4e7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:24.747206 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a7b5-7217-4431-9f19-a60c6db4e7e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:24.754724 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f727081-2aca-4f2b-8568-208bd1954cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:24.755509 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f727081-2aca-4f2b-8568-208bd1954cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:24.755945 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f727081-2aca-4f2b-8568-208bd1954cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:24.756799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f727081-2aca-4f2b-8568-208bd1954cc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:24.757844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f727081-2aca-4f2b-8568-208bd1954cc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:24.758281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f727081-2aca-4f2b-8568-208bd1954cc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:24.758389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1a7b5-7217-4431-9f19-a60c6db4e7e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:24.759911 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f727081-2aca-4f2b-8568-208bd1954cc0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:24.760043 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1a7b5-7217-4431-9f19-a60c6db4e7e8 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:24.762734 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:24.773563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 228/463] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:24.774938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402578f-f640-49f1-bbb1-744774a118eb None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:24.775219 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402578f-f640-49f1-bbb1-744774a118eb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:24.775482 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402578f-f640-49f1-bbb1-744774a118eb None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-0ca1a7b5-7217-4431-9f19-a60c6db4e7e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:24.777769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:24.778657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:24.779482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:24.779482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:24.779482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:24.780947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 229/464] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:24.781656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 236/465] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:24.781764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402578f-f640-49f1-bbb1-744774a118eb None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:24.787707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a7b5-7217-4431-9f19-a60c6db4e7e8 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:24.893229 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a7b5-7217-4431-9f19-a60c6db4e7e8 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:24.902999 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402578f-f640-49f1-bbb1-744774a118eb None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:24.930152 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91225e2a-3dc3-4315-adff-003812bd464d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:24.930899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91225e2a-3dc3-4315-adff-003812bd464d None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:24.931174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91225e2a-3dc3-4315-adff-003812bd464d None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:24.931449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91225e2a-3dc3-4315-adff-003812bd464d None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:24.932072 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91225e2a-3dc3-4315-adff-003812bd464d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:24.932518 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91225e2a-3dc3-4315-adff-003812bd464d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:24.933830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91225e2a-3dc3-4315-adff-003812bd464d None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:24.935356 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab534b86-510c-49bd-8495-6b92cb773bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:24.935957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab534b86-510c-49bd-8495-6b92cb773bc1 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:24.936233 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab534b86-510c-49bd-8495-6b92cb773bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:24.936497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab534b86-510c-49bd-8495-6b92cb773bc1 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:24.937133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab534b86-510c-49bd-8495-6b92cb773bc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:24.938596 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab534b86-510c-49bd-8495-6b92cb773bc1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:24.947555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 237/466] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:24.948570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be34a1-3c0d-42a8-bd52-3da2b53c385f None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:24.948846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be34a1-3c0d-42a8-bd52-3da2b53c385f None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:24.949114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be34a1-3c0d-42a8-bd52-3da2b53c385f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:24.966787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 230/467] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:18:24 2022] POST /identity/v3/users => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:18:24.968004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab534b86-510c-49bd-8495-6b92cb773bc1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:24.976049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-07be34a1-3c0d-42a8-bd52-3da2b53c385f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:24.977558 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-07be34a1-3c0d-42a8-bd52-3da2b53c385f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:25.009020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:25.044265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:25.063661 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 238/468] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:25.064955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be34a1-3c0d-42a8-bd52-3da2b53c385f None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:25.159512 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab534b86-510c-49bd-8495-6b92cb773bc1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:25.198375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8f655-828e-4299-b22a-446ba54b0b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:25.199042 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8f655-828e-4299-b22a-446ba54b0b26 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:25.199315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8f655-828e-4299-b22a-446ba54b0b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:25.199583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8f655-828e-4299-b22a-446ba54b0b26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:25.204161 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be34a1-3c0d-42a8-bd52-3da2b53c385f None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:25.210851 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8f655-828e-4299-b22a-446ba54b0b26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:25.212325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8f655-828e-4299-b22a-446ba54b0b26 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-3fc8f655-828e-4299-b22a-446ba54b0b26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:25.229549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:25.230444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:25.231339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:25.231339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:25.231339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:25.231339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 231/469] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:25.237740 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8f655-828e-4299-b22a-446ba54b0b26 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:25.247224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e535c-1586-4e4c-8d0c-3036535b619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:25.248260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e535c-1586-4e4c-8d0c-3036535b619e None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:25.248543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e535c-1586-4e4c-8d0c-3036535b619e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:25.248806 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e535c-1586-4e4c-8d0c-3036535b619e None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:25.249458 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e535c-1586-4e4c-8d0c-3036535b619e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:25.251346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e535c-1586-4e4c-8d0c-3036535b619e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:25.293874 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 239/470] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:18:25 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:18:25.333347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:25.362195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8f655-828e-4299-b22a-446ba54b0b26 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:25.407630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4f7c3-9dd5-442c-b1b6-96d2be65f920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:25.408639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4f7c3-9dd5-442c-b1b6-96d2be65f920 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:25.409108 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4f7c3-9dd5-442c-b1b6-96d2be65f920 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:25.409591 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4f7c3-9dd5-442c-b1b6-96d2be65f920 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:25.410466 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4f7c3-9dd5-442c-b1b6-96d2be65f920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:25.410995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4f7c3-9dd5-442c-b1b6-96d2be65f920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:25.412613 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4f7c3-9dd5-442c-b1b6-96d2be65f920 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:25.421419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e535c-1586-4e4c-8d0c-3036535b619e None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:25.428290 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 232/471] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:25.434448 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4f7c3-9dd5-442c-b1b6-96d2be65f920 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:25.481408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:25.548155 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4f7c3-9dd5-442c-b1b6-96d2be65f920 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:25.550724 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e535c-1586-4e4c-8d0c-3036535b619e None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:25.580531 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:25.588752 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f45e06-3a3b-4676-b3f1-6c343a34eeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:25.589576 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f45e06-3a3b-4676-b3f1-6c343a34eeb9 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:25.589680 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f45e06-3a3b-4676-b3f1-6c343a34eeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:25.589889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f45e06-3a3b-4676-b3f1-6c343a34eeb9 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:25.591235 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f45e06-3a3b-4676-b3f1-6c343a34eeb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:25.592668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f45e06-3a3b-4676-b3f1-6c343a34eeb9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:25.601803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68298d-e17e-41ea-82d6-96c0811b28b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:25.602831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68298d-e17e-41ea-82d6-96c0811b28b6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:25.603276 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68298d-e17e-41ea-82d6-96c0811b28b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:25.604293 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68298d-e17e-41ea-82d6-96c0811b28b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:25.622368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 233/472] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:18:25 2022] POST /identity/v3/users => generated 248 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:18:25.622368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563176-0590-48c9-a67c-313b7e85eeab None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:25.622368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563176-0590-48c9-a67c-313b7e85eeab None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:25.622368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563176-0590-48c9-a67c-313b7e85eeab None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:25.622368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 234/473] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:25.624591 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68298d-e17e-41ea-82d6-96c0811b28b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:25.626333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68298d-e17e-41ea-82d6-96c0811b28b6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:25.627845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9224ea-1147-4dde-80bd-173881455cdc None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:25.627845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9224ea-1147-4dde-80bd-173881455cdc None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:25.627845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9224ea-1147-4dde-80bd-173881455cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-9c68298d-e17e-41ea-82d6-96c0811b28b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:25.645772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:25.646819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:25.647784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:25.647784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:25.647784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:25.649122 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3d9224ea-1147-4dde-80bd-173881455cdc None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:25.650133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 240/474] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:25.655072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3d9224ea-1147-4dde-80bd-173881455cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:25.660321 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68298d-e17e-41ea-82d6-96c0811b28b6 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:25.779712 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 235/475] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:25.862976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68298d-e17e-41ea-82d6-96c0811b28b6 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:25.866599 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:25.899457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533268f1-9c4e-4930-8db9-2e51dfdbd980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:25.900126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533268f1-9c4e-4930-8db9-2e51dfdbd980 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:25.900394 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533268f1-9c4e-4930-8db9-2e51dfdbd980 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:25.900663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533268f1-9c4e-4930-8db9-2e51dfdbd980 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:25.901364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533268f1-9c4e-4930-8db9-2e51dfdbd980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:25.901673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533268f1-9c4e-4930-8db9-2e51dfdbd980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:25.903138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533268f1-9c4e-4930-8db9-2e51dfdbd980 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:25.915457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 241/476] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:25.924045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9224ea-1147-4dde-80bd-173881455cdc None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:26.031873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9224ea-1147-4dde-80bd-173881455cdc None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:26.052177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:26.070289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5eb0f-cb80-4d57-a6c1-c94f623c2f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:26.070996 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5eb0f-cb80-4d57-a6c1-c94f623c2f99 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:26.071427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5eb0f-cb80-4d57-a6c1-c94f623c2f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:26.071992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5eb0f-cb80-4d57-a6c1-c94f623c2f99 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:26.072640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5eb0f-cb80-4d57-a6c1-c94f623c2f99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:26.074082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5eb0f-cb80-4d57-a6c1-c94f623c2f99 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:26.103484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 236/477] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:18:25 2022] POST /identity/v3/users => generated 249 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:18:26.121635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533268f1-9c4e-4930-8db9-2e51dfdbd980 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:26.233048 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533268f1-9c4e-4930-8db9-2e51dfdbd980 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:26.273954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992a051-d047-4958-b54e-3e6c00ee4498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:26.274623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992a051-d047-4958-b54e-3e6c00ee4498 None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:26.274907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992a051-d047-4958-b54e-3e6c00ee4498 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:26.275180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992a051-d047-4958-b54e-3e6c00ee4498 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:26.275882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992a051-d047-4958-b54e-3e6c00ee4498 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:26.277389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992a051-d047-4958-b54e-3e6c00ee4498 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:26.299160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.sql.core [None req-6992a051-d047-4958-b54e-3e6c00ee4498 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 16:18:26.299160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 16:18:26.299160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [parameters: {'id': '10e0f71eb2b94115a698b66590b79d94', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 16:18:26.299160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81510) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-6992a051-d047-4958-b54e-3e6c00ee4498 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application conn.query(q) Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application result.read() Sep 09 16:18:26.318070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return ref.to_dict() Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 16:18:26.319084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application next(self.gen) Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application yield resource Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application next(self.gen) Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise self.value Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application session.commit() Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 16:18:26.320216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self.session.flush() Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application compat.raise_( Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 16:18:26.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 16:18:26.322062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application conn.query(q) Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application result.read() Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 16:18:26.322897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [parameters: {'id': '10e0f71eb2b94115a698b66590b79d94', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:26.323662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 16:18:26.326861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:26.327746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 242/478] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Sep 9 16:18:26 2022] POST /identity/v3/roles => generated 144 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:26.327809 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5eb0f-cb80-4d57-a6c1-c94f623c2f99 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:26.452803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5eb0f-cb80-4d57-a6c1-c94f623c2f99 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:26.486975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f618370-b6f8-4784-b3c3-ccd702c729ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:26.487613 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f618370-b6f8-4784-b3c3-ccd702c729ec None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:26.487943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f618370-b6f8-4784-b3c3-ccd702c729ec None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:26.488217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f618370-b6f8-4784-b3c3-ccd702c729ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:26.498857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f618370-b6f8-4784-b3c3-ccd702c729ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-4f618370-b6f8-4784-b3c3-ccd702c729ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:26.504179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:26.505034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:26.506598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:26.506598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:26.506598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:26.506598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:26.506598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:26.506598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:26.506598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:26.506598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:26.506598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:26.506598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:26.506598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 16:18:26.506598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:26.506598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 237/479] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:18:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:26.511368 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992a051-d047-4958-b54e-3e6c00ee4498 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:26.531222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:26.630566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992a051-d047-4958-b54e-3e6c00ee4498 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:26.672725 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc8867-435d-47fe-9e2d-662cfc0f191b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:26.673349 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc8867-435d-47fe-9e2d-662cfc0f191b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:26.673625 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc8867-435d-47fe-9e2d-662cfc0f191b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:26.673895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc8867-435d-47fe-9e2d-662cfc0f191b None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:26.674585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc8867-435d-47fe-9e2d-662cfc0f191b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:26.674888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc8867-435d-47fe-9e2d-662cfc0f191b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:26.676259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc8867-435d-47fe-9e2d-662cfc0f191b None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:26.689745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 243/480] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:27.658905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29040f-470a-49ed-93a5-91de75124de5 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:27.659469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29040f-470a-49ed-93a5-91de75124de5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:27.659469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29040f-470a-49ed-93a5-91de75124de5 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:27.661047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 238/481] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:27.670794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602dc3a-1c6f-4376-8c40-c749a4f260e1 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:27.671237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602dc3a-1c6f-4376-8c40-c749a4f260e1 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:27.671323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602dc3a-1c6f-4376-8c40-c749a4f260e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:27.695391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d602dc3a-1c6f-4376-8c40-c749a4f260e1 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:27.696578 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d602dc3a-1c6f-4376-8c40-c749a4f260e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:27.781103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 244/482] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:27.923415 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1146e-279b-4afe-b559-780016408a28 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:27.923597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1146e-279b-4afe-b559-780016408a28 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:27.923864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1146e-279b-4afe-b559-780016408a28 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:27.925445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 239/483] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:27.931104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c4eba-0ed8-4d1f-b748-9c55c47fccc7 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:27.931392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c4eba-0ed8-4d1f-b748-9c55c47fccc7 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:27.931668 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c4eba-0ed8-4d1f-b748-9c55c47fccc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:27.950419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-006c4eba-0ed8-4d1f-b748-9c55c47fccc7 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:27.951640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-006c4eba-0ed8-4d1f-b748-9c55c47fccc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:27.954667 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:28.048036 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 245/484] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:28.167258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1146e-279b-4afe-b559-780016408a28 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:28.213677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:28.302087 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1146e-279b-4afe-b559-780016408a28 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:28.351021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82ed4e-127a-4276-a429-464603610fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:28.351656 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82ed4e-127a-4276-a429-464603610fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:28.351932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82ed4e-127a-4276-a429-464603610fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:28.352195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82ed4e-127a-4276-a429-464603610fc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:28.362343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82ed4e-127a-4276-a429-464603610fc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-ab82ed4e-127a-4276-a429-464603610fc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:28.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:28.368083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:28.368897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:28.368897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:28.368897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:28.368897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:28.368897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:28.368897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:28.368897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:28.368897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:28.368897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:28.368897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:28.368897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:28.368897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:28.368897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 240/485] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:28.374301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c4eba-0ed8-4d1f-b748-9c55c47fccc7 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:28.456514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82ed4e-127a-4276-a429-464603610fc6 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:28.542689 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c4eba-0ed8-4d1f-b748-9c55c47fccc7 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:28.582721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9c6cb-7b91-4e1a-8cb6-8d84840a9c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:28.583438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9c6cb-7b91-4e1a-8cb6-8d84840a9c21 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:28.583715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9c6cb-7b91-4e1a-8cb6-8d84840a9c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:28.583982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9c6cb-7b91-4e1a-8cb6-8d84840a9c21 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:28.584709 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9c6cb-7b91-4e1a-8cb6-8d84840a9c21 None admin] RBAC: Adding query filter params (name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:28.585008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9c6cb-7b91-4e1a-8cb6-8d84840a9c21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:28.586697 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9c6cb-7b91-4e1a-8cb6-8d84840a9c21 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:28.589285 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:28.597083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82ed4e-127a-4276-a429-464603610fc6 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:28.597853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 246/486] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:28.599219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e2d50-9af8-4cd8-841f-b113b0737645 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:28.599501 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e2d50-9af8-4cd8-841f-b113b0737645 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:28.599776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e2d50-9af8-4cd8-841f-b113b0737645 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:28.605314 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 247/487] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:28.606171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e2d50-9af8-4cd8-841f-b113b0737645 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:28.608985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:28.634778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb894731-7048-4c12-85db-8ce23cd06b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:28.635416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb894731-7048-4c12-85db-8ce23cd06b6f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:28.635768 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb894731-7048-4c12-85db-8ce23cd06b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:28.636040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb894731-7048-4c12-85db-8ce23cd06b6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:28.647117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb894731-7048-4c12-85db-8ce23cd06b6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-fb894731-7048-4c12-85db-8ce23cd06b6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:28.652109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:28.653391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:28.654175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:28.654175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:28.654175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:28.654175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:28.654175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:28.654175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:28.654175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:28.654175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:28.654175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:28.654175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:28.654175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:28.654175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:28.654175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 241/488] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:28.659424 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0d8a7-98f5-48fa-86f6-4d80b409afed None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:28.660052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0d8a7-98f5-48fa-86f6-4d80b409afed None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:28.660052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0d8a7-98f5-48fa-86f6-4d80b409afed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:28.678543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-fdf0d8a7-98f5-48fa-86f6-4d80b409afed None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:28.679742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-fdf0d8a7-98f5-48fa-86f6-4d80b409afed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:28.722345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e2d50-9af8-4cd8-841f-b113b0737645 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:28.766212 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd2711-5480-4755-82ef-f2b7793166fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:28.767417 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd2711-5480-4755-82ef-f2b7793166fb None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:28.767698 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd2711-5480-4755-82ef-f2b7793166fb None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:28.770538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd2711-5480-4755-82ef-f2b7793166fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:28.772489 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 242/489] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:28.773360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0d8a7-98f5-48fa-86f6-4d80b409afed None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:28.788579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd2711-5480-4755-82ef-f2b7793166fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:28.790345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd2711-5480-4755-82ef-f2b7793166fb None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-6ddd2711-5480-4755-82ef-f2b7793166fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:28.820095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:28.821254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:28.822182 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:28.822182 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:28.822182 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:28.826139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 248/490] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:28.832405 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd2711-5480-4755-82ef-f2b7793166fb None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:28.939384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:28.953148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:28.960046 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0d8a7-98f5-48fa-86f6-4d80b409afed None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:28.995962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b6007-b36f-47d7-ad09-53e91dc5e500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:28.996816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b6007-b36f-47d7-ad09-53e91dc5e500 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:28.997231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b6007-b36f-47d7-ad09-53e91dc5e500 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:28.997665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b6007-b36f-47d7-ad09-53e91dc5e500 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:28.998469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b6007-b36f-47d7-ad09-53e91dc5e500 None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:28.998962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b6007-b36f-47d7-ad09-53e91dc5e500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:29.000584 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b6007-b36f-47d7-ad09-53e91dc5e500 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:29.009840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 243/491] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:29.015944 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7f6fe-152e-40a6-afe2-f78e993dfd5a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:29.016394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7f6fe-152e-40a6-afe2-f78e993dfd5a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:29.016824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7f6fe-152e-40a6-afe2-f78e993dfd5a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:29.018315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 244/492] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:29.019242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7f6fe-152e-40a6-afe2-f78e993dfd5a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:29.036872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd2711-5480-4755-82ef-f2b7793166fb None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:29.071590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d870ba-97b6-4130-beab-7b96295aca74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:29.072117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d870ba-97b6-4130-beab-7b96295aca74 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:29.072391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d870ba-97b6-4130-beab-7b96295aca74 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:29.072657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d870ba-97b6-4130-beab-7b96295aca74 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:29.073301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d870ba-97b6-4130-beab-7b96295aca74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:29.073596 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d870ba-97b6-4130-beab-7b96295aca74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:29.075059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d870ba-97b6-4130-beab-7b96295aca74 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:29.093933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 249/493] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:29.095455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719393c1-cddb-4049-bd80-d32a835e842b None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:29.095737 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719393c1-cddb-4049-bd80-d32a835e842b None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:29.095999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719393c1-cddb-4049-bd80-d32a835e842b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:29.116687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-719393c1-cddb-4049-bd80-d32a835e842b None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:29.117872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-719393c1-cddb-4049-bd80-d32a835e842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:29.146053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7f6fe-152e-40a6-afe2-f78e993dfd5a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:29.184180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2c448-2c83-48d0-9d27-8c526367b7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:29.184806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2c448-2c83-48d0-9d27-8c526367b7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:29.185079 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2c448-2c83-48d0-9d27-8c526367b7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:29.185341 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2c448-2c83-48d0-9d27-8c526367b7a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:29.192276 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 250/494] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:29.193279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719393c1-cddb-4049-bd80-d32a835e842b None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:29.199229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2c448-2c83-48d0-9d27-8c526367b7a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:29.200802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2c448-2c83-48d0-9d27-8c526367b7a4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:29.210058 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-caa2c448-2c83-48d0-9d27-8c526367b7a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:29.221050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:29.221957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:29.222860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:29.222860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:29.222860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:29.230106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 245/495] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:29.230106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2c448-2c83-48d0-9d27-8c526367b7a4 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:29.248791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:29.305506 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719393c1-cddb-4049-bd80-d32a835e842b None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:29.339947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2c448-2c83-48d0-9d27-8c526367b7a4 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:29.344767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac27a55-b1c8-4763-8502-1eca4392cb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:29.345414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac27a55-b1c8-4763-8502-1eca4392cb23 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:29.345695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac27a55-b1c8-4763-8502-1eca4392cb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:29.345967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac27a55-b1c8-4763-8502-1eca4392cb23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:29.357689 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac27a55-b1c8-4763-8502-1eca4392cb23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-1ac27a55-b1c8-4763-8502-1eca4392cb23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:29.363481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:29.364363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:29.365137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:29.365137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:29.365137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:29.365137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:29.365137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:29.365137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:29.365137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:29.365137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:29.365137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:29.365137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:29.365137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:29.365137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:29.369824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 251/496] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:29.370698 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac27a55-b1c8-4763-8502-1eca4392cb23 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:29.384280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef8077-75f3-4472-8125-9302cfeb8cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:29.385294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef8077-75f3-4472-8125-9302cfeb8cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:29.385771 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef8077-75f3-4472-8125-9302cfeb8cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:29.386258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef8077-75f3-4472-8125-9302cfeb8cb9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:29.398064 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef8077-75f3-4472-8125-9302cfeb8cb9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:29.400204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef8077-75f3-4472-8125-9302cfeb8cb9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-96ef8077-75f3-4472-8125-9302cfeb8cb9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:29.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:29.416841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:29.417707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:29.417707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:29.417707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:29.417707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:29.417707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:29.417707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:29.417707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:29.417707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:29.417707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:18:29.417707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:29.420182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 246/497] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Sep 9 16:18:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:29.421297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef8077-75f3-4472-8125-9302cfeb8cb9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:29.483540 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:29.495616 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac27a55-b1c8-4763-8502-1eca4392cb23 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:29.498616 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:29.530157 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef8077-75f3-4472-8125-9302cfeb8cb9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:29.531683 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da49879-e23f-4838-a6eb-e1a3f46b150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:29.532300 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da49879-e23f-4838-a6eb-e1a3f46b150f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:29.532576 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da49879-e23f-4838-a6eb-e1a3f46b150f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:29.532842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da49879-e23f-4838-a6eb-e1a3f46b150f None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:29.533482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da49879-e23f-4838-a6eb-e1a3f46b150f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:29.533782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da49879-e23f-4838-a6eb-e1a3f46b150f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:29.535194 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da49879-e23f-4838-a6eb-e1a3f46b150f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:29.546621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 252/498] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:29.547412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da49879-e23f-4838-a6eb-e1a3f46b150f None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:29.564052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2197e-093a-4226-a2f0-6de45aeaa14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:29.564663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2197e-093a-4226-a2f0-6de45aeaa14c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:29.564941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2197e-093a-4226-a2f0-6de45aeaa14c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:29.565244 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2197e-093a-4226-a2f0-6de45aeaa14c None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:29.565903 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2197e-093a-4226-a2f0-6de45aeaa14c None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:29.566560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2197e-093a-4226-a2f0-6de45aeaa14c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:29.567981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2197e-093a-4226-a2f0-6de45aeaa14c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:29.575728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 247/499] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:29.576506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2197e-093a-4226-a2f0-6de45aeaa14c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:29.645929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da49879-e23f-4838-a6eb-e1a3f46b150f None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:29.673093 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2197e-093a-4226-a2f0-6de45aeaa14c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:29.678281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879759f-a716-4c83-b483-f2dcf04c7ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:29.678914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879759f-a716-4c83-b483-f2dcf04c7ecb None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:29.679192 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879759f-a716-4c83-b483-f2dcf04c7ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:29.679459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879759f-a716-4c83-b483-f2dcf04c7ecb None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:29.680180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879759f-a716-4c83-b483-f2dcf04c7ecb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:29.680476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879759f-a716-4c83-b483-f2dcf04c7ecb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:29.681938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879759f-a716-4c83-b483-f2dcf04c7ecb None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:29.692275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:29.707102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212a36c-ff90-44de-84c3-72fc78cdb951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:29.707726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212a36c-ff90-44de-84c3-72fc78cdb951 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:29.708001 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212a36c-ff90-44de-84c3-72fc78cdb951 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:29.708269 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212a36c-ff90-44de-84c3-72fc78cdb951 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:29.718570 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212a36c-ff90-44de-84c3-72fc78cdb951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:29.723299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 253/500] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Sep 9 16:18:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:29.723299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393b8e2-d0bd-4a2d-9c7f-4bc30055e892 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:29.723299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393b8e2-d0bd-4a2d-9c7f-4bc30055e892 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:29.723299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393b8e2-d0bd-4a2d-9c7f-4bc30055e892 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:29.723299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 254/501] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:29.723299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393b8e2-d0bd-4a2d-9c7f-4bc30055e892 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c212a36c-ff90-44de-84c3-72fc78cdb951 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:29.725139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:29.726045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:29.726865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:29.726865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:29.726865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:29.726865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:29.726865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:29.726865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:29.726865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:29.726865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:29.726865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:29.726865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:29.726865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:29.726865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:29.726865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 248/502] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:29.728205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212a36c-ff90-44de-84c3-72fc78cdb951 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:29.747526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:29.822035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393b8e2-d0bd-4a2d-9c7f-4bc30055e892 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:29.823820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212a36c-ff90-44de-84c3-72fc78cdb951 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:29.854349 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94b69f-ab25-4dab-aa47-8520eb53ca26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:29.855000 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94b69f-ab25-4dab-aa47-8520eb53ca26 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:29.855275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94b69f-ab25-4dab-aa47-8520eb53ca26 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:29.855539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94b69f-ab25-4dab-aa47-8520eb53ca26 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:29.855679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1022e-dbb3-40c1-b72e-6af93c8f296a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:29.856323 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1022e-dbb3-40c1-b72e-6af93c8f296a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:29.856579 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1022e-dbb3-40c1-b72e-6af93c8f296a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:29.856854 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1022e-dbb3-40c1-b72e-6af93c8f296a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:29.862394 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94b69f-ab25-4dab-aa47-8520eb53ca26 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:29.863847 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94b69f-ab25-4dab-aa47-8520eb53ca26 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:29.867887 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1022e-dbb3-40c1-b72e-6af93c8f296a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:29.869313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1022e-dbb3-40c1-b72e-6af93c8f296a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-1b94b69f-ab25-4dab-aa47-8520eb53ca26 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:29.876973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:29.877853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:29.878762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:29.878762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:29.878762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:29.878762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:29.878762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:29.878762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:29.878762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:29.878762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:29.878762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 16:18:29.878762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:29.878762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 255/503] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:29.879345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94b69f-ab25-4dab-aa47-8520eb53ca26 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-96f1022e-dbb3-40c1-b72e-6af93c8f296a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:29.887964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:29.888848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:29.889660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:29.889660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:29.889660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:29.889660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 249/504] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:29.890336 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e0b83-e8de-4de5-b75c-68a81eecc910 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:29.890487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e0b83-e8de-4de5-b75c-68a81eecc910 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:29.890787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e0b83-e8de-4de5-b75c-68a81eecc910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:29.908525 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ee9e0b83-e8de-4de5-b75c-68a81eecc910 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:29.909640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ee9e0b83-e8de-4de5-b75c-68a81eecc910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:29.935624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:29.960852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:29.975884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 250/505] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:29.976531 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e0b83-e8de-4de5-b75c-68a81eecc910 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:29.986817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94b69f-ab25-4dab-aa47-8520eb53ca26 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:30.022682 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364d8aa-58eb-4435-ac84-0dc1c54d0f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:30.023293 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d8aa-58eb-4435-ac84-0dc1c54d0f13 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:30.023568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d8aa-58eb-4435-ac84-0dc1c54d0f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:30.023831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d8aa-58eb-4435-ac84-0dc1c54d0f13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:30.092412 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e0b83-e8de-4de5-b75c-68a81eecc910 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:30.095954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364d8aa-58eb-4435-ac84-0dc1c54d0f13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:30.096754 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364d8aa-58eb-4435-ac84-0dc1c54d0f13 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:30.129070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68b1dc-5ccf-4104-9ef6-82f6fa4dc080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:30.129677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68b1dc-5ccf-4104-9ef6-82f6fa4dc080 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:30.129956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68b1dc-5ccf-4104-9ef6-82f6fa4dc080 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:30.130221 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68b1dc-5ccf-4104-9ef6-82f6fa4dc080 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:30.130909 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68b1dc-5ccf-4104-9ef6-82f6fa4dc080 None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:30.131207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68b1dc-5ccf-4104-9ef6-82f6fa4dc080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:30.132597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68b1dc-5ccf-4104-9ef6-82f6fa4dc080 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:30.145075 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 251/506] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:30.145075 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68b1dc-5ccf-4104-9ef6-82f6fa4dc080 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:30.169214 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:30.182345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:30.206542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 256/507] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:30.207368 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364d8aa-58eb-4435-ac84-0dc1c54d0f13 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:30.249373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68b1dc-5ccf-4104-9ef6-82f6fa4dc080 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:30.282240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b57b2-bf87-429c-b785-b13d79ee56cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:30.282867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b57b2-bf87-429c-b785-b13d79ee56cf None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:30.283139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b57b2-bf87-429c-b785-b13d79ee56cf None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:30.283399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b57b2-bf87-429c-b785-b13d79ee56cf None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:30.284108 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b57b2-bf87-429c-b785-b13d79ee56cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:30.284406 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b57b2-bf87-429c-b785-b13d79ee56cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:30.285848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b57b2-bf87-429c-b785-b13d79ee56cf None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:30.303399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364d8aa-58eb-4435-ac84-0dc1c54d0f13 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:30.321054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 252/508] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Sep 9 16:18:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:30.321860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b57b2-bf87-429c-b785-b13d79ee56cf None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:30.338317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81baf1-ad65-4d4b-b77c-7de4d51d8d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:30.339003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81baf1-ad65-4d4b-b77c-7de4d51d8d0a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:30.339276 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81baf1-ad65-4d4b-b77c-7de4d51d8d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:30.339599 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81baf1-ad65-4d4b-b77c-7de4d51d8d0a None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:30.340252 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81baf1-ad65-4d4b-b77c-7de4d51d8d0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:30.340553 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81baf1-ad65-4d4b-b77c-7de4d51d8d0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:30.341941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81baf1-ad65-4d4b-b77c-7de4d51d8d0a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:30.354364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 257/509] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:30.355186 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81baf1-ad65-4d4b-b77c-7de4d51d8d0a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:30.400955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:30.402376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:30.421926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b57b2-bf87-429c-b785-b13d79ee56cf None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:30.457343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc99752-b169-4917-a410-a3a1d174bbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:30.458089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc99752-b169-4917-a410-a3a1d174bbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:30.458219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81baf1-ad65-4d4b-b77c-7de4d51d8d0a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:30.458483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc99752-b169-4917-a410-a3a1d174bbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:30.458853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc99752-b169-4917-a410-a3a1d174bbd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:30.469882 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc99752-b169-4917-a410-a3a1d174bbd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:30.471388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc99752-b169-4917-a410-a3a1d174bbd0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-afc99752-b169-4917-a410-a3a1d174bbd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:30.485914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:30.487686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:30.489292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:30.489292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:30.489292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:30.489292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 253/510] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:30.489292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc99752-b169-4917-a410-a3a1d174bbd0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:30.499879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4762cf-1e99-4fbd-9f24-af1a7a07a76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:30.500528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4762cf-1e99-4fbd-9f24-af1a7a07a76c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:30.500796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4762cf-1e99-4fbd-9f24-af1a7a07a76c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:30.501056 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4762cf-1e99-4fbd-9f24-af1a7a07a76c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:30.511728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4762cf-1e99-4fbd-9f24-af1a7a07a76c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:30.513196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4762cf-1e99-4fbd-9f24-af1a7a07a76c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-cc4762cf-1e99-4fbd-9f24-af1a7a07a76c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:30.527384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:30.528430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:30.529265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:30.529265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:30.529265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:30.529265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 258/511] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:30.529497 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4762cf-1e99-4fbd-9f24-af1a7a07a76c None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:30.589174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc99752-b169-4917-a410-a3a1d174bbd0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:30.621709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c9dce-ec88-4b32-bdc2-f8b4e45376b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:30.622293 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:30.622394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c9dce-ec88-4b32-bdc2-f8b4e45376b8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:30.622627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c9dce-ec88-4b32-bdc2-f8b4e45376b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:30.622898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c9dce-ec88-4b32-bdc2-f8b4e45376b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:30.625230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:30.629394 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4762cf-1e99-4fbd-9f24-af1a7a07a76c None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:30.634901 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c9dce-ec88-4b32-bdc2-f8b4e45376b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:30.636374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c9dce-ec88-4b32-bdc2-f8b4e45376b8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-197c9dce-ec88-4b32-bdc2-f8b4e45376b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:30.651034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:30.651912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:30.652766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:30.652766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:30.652766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:30.652766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 254/512] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:30.653358 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c9dce-ec88-4b32-bdc2-f8b4e45376b8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:30.663360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075ae33-3db5-4e34-8e97-3feddd6c5a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:30.663965 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075ae33-3db5-4e34-8e97-3feddd6c5a30 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:30.664238 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075ae33-3db5-4e34-8e97-3feddd6c5a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:30.664488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075ae33-3db5-4e34-8e97-3feddd6c5a30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:30.730857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075ae33-3db5-4e34-8e97-3feddd6c5a30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:30.732271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075ae33-3db5-4e34-8e97-3feddd6c5a30 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:30.751786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c9dce-ec88-4b32-bdc2-f8b4e45376b8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:30.784930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2403da-c70a-497a-aa8e-c711a4922ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:30.785556 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2403da-c70a-497a-aa8e-c711a4922ccc None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:30.785832 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2403da-c70a-497a-aa8e-c711a4922ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:30.786096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2403da-c70a-497a-aa8e-c711a4922ccc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:30.793265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2403da-c70a-497a-aa8e-c711a4922ccc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:30.794841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2403da-c70a-497a-aa8e-c711a4922ccc None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-8b2403da-c70a-497a-aa8e-c711a4922ccc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:30.807728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:30.808703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:30.809563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:30.809563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:30.809563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:30.809563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:30.809563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:30.809563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:30.809563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:30.809563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:30.809563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 16:18:30.809563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:30.809563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 255/513] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:30.809563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2403da-c70a-497a-aa8e-c711a4922ccc None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:30.830432 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:30.838157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 259/514] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:30.839207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075ae33-3db5-4e34-8e97-3feddd6c5a30 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:30.858546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:30.910209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2403da-c70a-497a-aa8e-c711a4922ccc None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:30.942129 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075ae33-3db5-4e34-8e97-3feddd6c5a30 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:30.942705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c08df-d9f5-43a8-9899-9b4ae1adcf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:30.943313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c08df-d9f5-43a8-9899-9b4ae1adcf20 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:30.943588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c08df-d9f5-43a8-9899-9b4ae1adcf20 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:30.943940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c08df-d9f5-43a8-9899-9b4ae1adcf20 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:30.944579 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c08df-d9f5-43a8-9899-9b4ae1adcf20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:30.944876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c08df-d9f5-43a8-9899-9b4ae1adcf20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:30.946266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c08df-d9f5-43a8-9899-9b4ae1adcf20 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:30.957698 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 256/515] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:30.958484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c08df-d9f5-43a8-9899-9b4ae1adcf20 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:30.976335 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eaa497-f416-4591-8ede-36d1de9945d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:30.976948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaa497-f416-4591-8ede-36d1de9945d6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:30.977247 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaa497-f416-4591-8ede-36d1de9945d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:30.977491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaa497-f416-4591-8ede-36d1de9945d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:30.978132 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eaa497-f416-4591-8ede-36d1de9945d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:30.978485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eaa497-f416-4591-8ede-36d1de9945d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:30.979901 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eaa497-f416-4591-8ede-36d1de9945d6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:30.990808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 260/516] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:30.991528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eaa497-f416-4591-8ede-36d1de9945d6 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:31.045382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:31.054203 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c08df-d9f5-43a8-9899-9b4ae1adcf20 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:31.061678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:31.093892 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bd753-43ca-48f4-8a39-3cde60264eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:31.094611 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bd753-43ca-48f4-8a39-3cde60264eeb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:31.094888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bd753-43ca-48f4-8a39-3cde60264eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:31.095154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bd753-43ca-48f4-8a39-3cde60264eeb None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:31.095786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7bd753-43ca-48f4-8a39-3cde60264eeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:31.096079 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7bd753-43ca-48f4-8a39-3cde60264eeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:31.096751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eaa497-f416-4591-8ede-36d1de9945d6 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:31.097434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7bd753-43ca-48f4-8a39-3cde60264eeb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:31.112946 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 257/517] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:31.112946 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bd753-43ca-48f4-8a39-3cde60264eeb None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:31.129471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057032b-7f19-4b53-8e15-bab689816a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:31.130083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057032b-7f19-4b53-8e15-bab689816a08 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:31.130352 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057032b-7f19-4b53-8e15-bab689816a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:31.130653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057032b-7f19-4b53-8e15-bab689816a08 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:31.131472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057032b-7f19-4b53-8e15-bab689816a08 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:31.131776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057032b-7f19-4b53-8e15-bab689816a08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:31.133208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057032b-7f19-4b53-8e15-bab689816a08 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:31.150393 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 261/518] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Sep 9 16:18:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:31.151211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057032b-7f19-4b53-8e15-bab689816a08 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:31.208254 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bd753-43ca-48f4-8a39-3cde60264eeb None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:31.241630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70710f04-e59f-44db-9af6-9c49ab457212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:31.242184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70710f04-e59f-44db-9af6-9c49ab457212 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:31.242456 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70710f04-e59f-44db-9af6-9c49ab457212 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:31.242756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70710f04-e59f-44db-9af6-9c49ab457212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:31.249342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057032b-7f19-4b53-8e15-bab689816a08 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:31.253962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70710f04-e59f-44db-9af6-9c49ab457212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:31.255477 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70710f04-e59f-44db-9af6-9c49ab457212 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:31.262026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-70710f04-e59f-44db-9af6-9c49ab457212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:31.270055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:31.271110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:31.272027 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:31.272027 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:31.272027 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:31.272027 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 258/519] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:31.272243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70710f04-e59f-44db-9af6-9c49ab457212 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:31.287256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47489789-a5b6-42ae-be42-733dd9ac3882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:31.287915 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47489789-a5b6-42ae-be42-733dd9ac3882 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:31.288207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47489789-a5b6-42ae-be42-733dd9ac3882 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:31.288470 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47489789-a5b6-42ae-be42-733dd9ac3882 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:31.291368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:31.295704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47489789-a5b6-42ae-be42-733dd9ac3882 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:31.297154 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47489789-a5b6-42ae-be42-733dd9ac3882 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-47489789-a5b6-42ae-be42-733dd9ac3882 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:31.308427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:31.309503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:31.310623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:31.310623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:31.310623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:31.310623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:31.310623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:31.310623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:31.310623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:31.310623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:31.310623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 16:18:31.310623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:31.310623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 262/520] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:31.311313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47489789-a5b6-42ae-be42-733dd9ac3882 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:31.375102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70710f04-e59f-44db-9af6-9c49ab457212 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:31.406477 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47489789-a5b6-42ae-be42-733dd9ac3882 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:31.409803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3fedc-1ae1-45d9-babe-253ce1524f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:31.410439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3fedc-1ae1-45d9-babe-253ce1524f99 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:31.410746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3fedc-1ae1-45d9-babe-253ce1524f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:31.411016 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3fedc-1ae1-45d9-babe-253ce1524f99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:31.417811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3fedc-1ae1-45d9-babe-253ce1524f99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:31.419300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3fedc-1ae1-45d9-babe-253ce1524f99 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-72b3fedc-1ae1-45d9-babe-253ce1524f99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:31.431741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:31.432662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:31.433668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:31.433668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:31.433668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:31.433668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:31.433668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:31.433668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:31.433668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:31.433668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:31.433668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 259/521] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:31.433668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3fedc-1ae1-45d9-babe-253ce1524f99 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:31.444190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c32d76-df61-4faa-a65f-fe7995392d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:31.444802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c32d76-df61-4faa-a65f-fe7995392d28 None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:31.445077 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c32d76-df61-4faa-a65f-fe7995392d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:31.445346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c32d76-df61-4faa-a65f-fe7995392d28 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:31.445970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c32d76-df61-4faa-a65f-fe7995392d28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:31.447441 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c32d76-df61-4faa-a65f-fe7995392d28 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:31.486536 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 263/522] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:18:31 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:18:31.486536 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c32d76-df61-4faa-a65f-fe7995392d28 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:31.503483 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:31.531410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:31.555673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3fedc-1ae1-45d9-babe-253ce1524f99 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:31.587399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c32d76-df61-4faa-a65f-fe7995392d28 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:31.593785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55c632-418e-4e7b-a17a-73e0f62dfa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:31.594548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55c632-418e-4e7b-a17a-73e0f62dfa9c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:31.595029 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55c632-418e-4e7b-a17a-73e0f62dfa9c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:31.595392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55c632-418e-4e7b-a17a-73e0f62dfa9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:31.624512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2a050-a7f8-43a9-8dbd-9ffca27bb5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:31.625127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2a050-a7f8-43a9-8dbd-9ffca27bb5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:31.625409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2a050-a7f8-43a9-8dbd-9ffca27bb5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:31.625683 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2a050-a7f8-43a9-8dbd-9ffca27bb5e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:31.626349 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2a050-a7f8-43a9-8dbd-9ffca27bb5e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:31.626688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2a050-a7f8-43a9-8dbd-9ffca27bb5e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:31.628084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2a050-a7f8-43a9-8dbd-9ffca27bb5e2 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:31.641411 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 264/523] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:31.642274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2a050-a7f8-43a9-8dbd-9ffca27bb5e2 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:31.676562 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55c632-418e-4e7b-a17a-73e0f62dfa9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:31.678057 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55c632-418e-4e7b-a17a-73e0f62dfa9c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:31.745426 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:31.759573 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2a050-a7f8-43a9-8dbd-9ffca27bb5e2 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:31.764208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:31.794293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 260/524] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:31.794293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55c632-418e-4e7b-a17a-73e0f62dfa9c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:31.803437 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce43515-692a-429d-b06d-5335f564cdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:31.804070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce43515-692a-429d-b06d-5335f564cdea None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:31.804341 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce43515-692a-429d-b06d-5335f564cdea None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:31.804633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce43515-692a-429d-b06d-5335f564cdea None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:31.805337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce43515-692a-429d-b06d-5335f564cdea None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:31.805636 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce43515-692a-429d-b06d-5335f564cdea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:31.807118 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce43515-692a-429d-b06d-5335f564cdea None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:31.826084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 265/525] 10.222.0.23 () {56 vars in 1139 bytes} [Fri Sep 9 16:18:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:31.826961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce43515-692a-429d-b06d-5335f564cdea None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:31.904551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55c632-418e-4e7b-a17a-73e0f62dfa9c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:31.935648 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce43515-692a-429d-b06d-5335f564cdea None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:31.937880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01684ed-bd82-4575-88eb-e18bbd28b3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:31.938853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01684ed-bd82-4575-88eb-e18bbd28b3ee None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:31.938853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01684ed-bd82-4575-88eb-e18bbd28b3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:31.939037 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01684ed-bd82-4575-88eb-e18bbd28b3ee None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:31.939716 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01684ed-bd82-4575-88eb-e18bbd28b3ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:31.940008 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01684ed-bd82-4575-88eb-e18bbd28b3ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:31.941806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01684ed-bd82-4575-88eb-e18bbd28b3ee None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:31.964124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 261/526] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:31.964124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01684ed-bd82-4575-88eb-e18bbd28b3ee None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:31.977275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f938c-d80f-42a1-8042-9b671a36056a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:31.978007 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f938c-d80f-42a1-8042-9b671a36056a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:31.978354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f938c-d80f-42a1-8042-9b671a36056a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:31.978794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f938c-d80f-42a1-8042-9b671a36056a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:31.986003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f938c-d80f-42a1-8042-9b671a36056a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:31.987613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f938c-d80f-42a1-8042-9b671a36056a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-9d5f938c-d80f-42a1-8042-9b671a36056a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:32.000438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:32.001480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:32.002659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:32.002659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:32.002659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:32.002659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:32.002659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:32.002659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:32.002659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:32.002659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:32.003554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 266/527] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:32.004372 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f938c-d80f-42a1-8042-9b671a36056a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:32.007981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:32.016957 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:32.069334 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01684ed-bd82-4575-88eb-e18bbd28b3ee None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:32.106848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04ad9e-4e37-4490-affd-3b267f2aac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:32.107552 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04ad9e-4e37-4490-affd-3b267f2aac87 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:32.107831 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04ad9e-4e37-4490-affd-3b267f2aac87 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:32.108095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04ad9e-4e37-4490-affd-3b267f2aac87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:32.118122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f938c-d80f-42a1-8042-9b671a36056a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:32.119133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04ad9e-4e37-4490-affd-3b267f2aac87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:32.121007 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04ad9e-4e37-4490-affd-3b267f2aac87 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-3d04ad9e-4e37-4490-affd-3b267f2aac87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:32.137621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:32.138790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:32.139834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:32.139834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:32.139834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:32.141172 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 262/528] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:32.142071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04ad9e-4e37-4490-affd-3b267f2aac87 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:32.158302 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31449d2-d451-4893-8c45-4897f00cdb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:32.158962 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31449d2-d451-4893-8c45-4897f00cdb5b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:32.159234 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31449d2-d451-4893-8c45-4897f00cdb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:32.159512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31449d2-d451-4893-8c45-4897f00cdb5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:32.228064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31449d2-d451-4893-8c45-4897f00cdb5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:32.229540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31449d2-d451-4893-8c45-4897f00cdb5b None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:32.236323 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:32.237396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:32.247929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04ad9e-4e37-4490-affd-3b267f2aac87 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:32.280650 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b60d6b-7293-48d3-b0d4-852200945504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:32.281282 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b60d6b-7293-48d3-b0d4-852200945504 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:32.281626 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b60d6b-7293-48d3-b0d4-852200945504 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:32.281963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b60d6b-7293-48d3-b0d4-852200945504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:32.282807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b60d6b-7293-48d3-b0d4-852200945504 None admin] RBAC: Adding query filter params (role.id=2b68b618a1b846f69bbe9aa08852a66f, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=152a41ad598b4b01b1881d928c53c848) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:32.283181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b60d6b-7293-48d3-b0d4-852200945504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:32.284682 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b60d6b-7293-48d3-b0d4-852200945504 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:32.290473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 263/529] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:18:32 2022] GET /identity/v3/role_assignments?user.id=152a41ad598b4b01b1881d928c53c848&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=2b68b618a1b846f69bbe9aa08852a66f => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:32.291365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b60d6b-7293-48d3-b0d4-852200945504 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:32.349660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 267/530] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:32.350930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31449d2-d451-4893-8c45-4897f00cdb5b None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:32.457125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b60d6b-7293-48d3-b0d4-852200945504 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:32.469913 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31449d2-d451-4893-8c45-4897f00cdb5b None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:32.479559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:32.504138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db869e90-d508-4eae-9c31-a918f9a9cf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:32.504919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db869e90-d508-4eae-9c31-a918f9a9cf34 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:32.505292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db869e90-d508-4eae-9c31-a918f9a9cf34 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:32.505659 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db869e90-d508-4eae-9c31-a918f9a9cf34 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:32.506571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db869e90-d508-4eae-9c31-a918f9a9cf34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:32.506940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db869e90-d508-4eae-9c31-a918f9a9cf34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:32.508437 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db869e90-d508-4eae-9c31-a918f9a9cf34 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:32.511340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59c2ce-f94f-405d-9815-24ca9a56bd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:32.511949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59c2ce-f94f-405d-9815-24ca9a56bd97 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:32.512224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59c2ce-f94f-405d-9815-24ca9a56bd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:32.512492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59c2ce-f94f-405d-9815-24ca9a56bd97 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:32.513126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59c2ce-f94f-405d-9815-24ca9a56bd97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:32.513421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59c2ce-f94f-405d-9815-24ca9a56bd97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:32.518557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59c2ce-f94f-405d-9815-24ca9a56bd97 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:32.525752 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 264/531] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:32.525752 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db869e90-d508-4eae-9c31-a918f9a9cf34 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:32.535751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 268/532] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:32.536950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59c2ce-f94f-405d-9815-24ca9a56bd97 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:32.537423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:32.629169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db869e90-d508-4eae-9c31-a918f9a9cf34 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:32.661325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59c2ce-f94f-405d-9815-24ca9a56bd97 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:32.664671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080824c-51f1-45a4-b513-762a82db4f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:32.665080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080824c-51f1-45a4-b513-762a82db4f9b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:32.665447 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080824c-51f1-45a4-b513-762a82db4f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:32.665872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080824c-51f1-45a4-b513-762a82db4f9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:32.676360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080824c-51f1-45a4-b513-762a82db4f9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:32.677989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080824c-51f1-45a4-b513-762a82db4f9b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-3080824c-51f1-45a4-b513-762a82db4f9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:32.697714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:32.698886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:32.699910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:32.699910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:32.699910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:32.702163 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46d847-50fc-4e6e-a54e-f7068480c9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:32.703063 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46d847-50fc-4e6e-a54e-f7068480c9dc None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:32.703480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46d847-50fc-4e6e-a54e-f7068480c9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:32.703865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46d847-50fc-4e6e-a54e-f7068480c9dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:32.704760 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46d847-50fc-4e6e-a54e-f7068480c9dc None admin] RBAC: Adding query filter params (role.id=2b68b618a1b846f69bbe9aa08852a66f, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=62a2fb70c9ae4f4b81ac4de513b868cc) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:32.705183 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46d847-50fc-4e6e-a54e-f7068480c9dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:32.705792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 265/533] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:32.706743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080824c-51f1-45a4-b513-762a82db4f9b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:32.707487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46d847-50fc-4e6e-a54e-f7068480c9dc None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:32.722681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 269/534] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:18:32 2022] GET /identity/v3/role_assignments?user.id=62a2fb70c9ae4f4b81ac4de513b868cc&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=2b68b618a1b846f69bbe9aa08852a66f => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:32.724797 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46d847-50fc-4e6e-a54e-f7068480c9dc None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:32.806952 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:32.820442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:32.894723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080824c-51f1-45a4-b513-762a82db4f9b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:32.918405 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46d847-50fc-4e6e-a54e-f7068480c9dc None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:32.959548 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ba390-24ae-4aaa-a3a5-46d04fe8dac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:32.960169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ba390-24ae-4aaa-a3a5-46d04fe8dac1 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:32.960446 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ba390-24ae-4aaa-a3a5-46d04fe8dac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:32.960718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ba390-24ae-4aaa-a3a5-46d04fe8dac1 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:32.961367 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ba390-24ae-4aaa-a3a5-46d04fe8dac1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:32.962371 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ba390-24ae-4aaa-a3a5-46d04fe8dac1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:32.963750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ba390-24ae-4aaa-a3a5-46d04fe8dac1 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:32.978116 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b640d8-9d2f-43c0-bf45-85de9ecaf3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:32.978779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b640d8-9d2f-43c0-bf45-85de9ecaf3aa None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:32.979058 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b640d8-9d2f-43c0-bf45-85de9ecaf3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:32.979314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b640d8-9d2f-43c0-bf45-85de9ecaf3aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:32.991023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 270/535] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:32.991693 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b640d8-9d2f-43c0-bf45-85de9ecaf3aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:32.993236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b640d8-9d2f-43c0-bf45-85de9ecaf3aa None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:32.995989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ba390-24ae-4aaa-a3a5-46d04fe8dac1 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-e7b640d8-9d2f-43c0-bf45-85de9ecaf3aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:33.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:33.009502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:33.010988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:33.010988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:33.010988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:33.010988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:33.010988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:33.010988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:33.010988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:33.010988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:33.010988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 266/536] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:33.015395 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b640d8-9d2f-43c0-bf45-85de9ecaf3aa None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:33.168564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b640d8-9d2f-43c0-bf45-85de9ecaf3aa None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:33.186934 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:33.204364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ba390-24ae-4aaa-a3a5-46d04fe8dac1 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:33.216912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:33.232443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bcaa3c-c6bd-4c1d-bdca-f9fc04004f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:33.233300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcaa3c-c6bd-4c1d-bdca-f9fc04004f51 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:33.233740 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcaa3c-c6bd-4c1d-bdca-f9fc04004f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:33.234143 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcaa3c-c6bd-4c1d-bdca-f9fc04004f51 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:33.235201 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bcaa3c-c6bd-4c1d-bdca-f9fc04004f51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:33.237141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bcaa3c-c6bd-4c1d-bdca-f9fc04004f51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:33.238865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bcaa3c-c6bd-4c1d-bdca-f9fc04004f51 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:33.264473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 267/537] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:33.268223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bcaa3c-c6bd-4c1d-bdca-f9fc04004f51 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:33.273417 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1089d73-cb6d-48ca-8218-3041fcce1836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:33.274957 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1089d73-cb6d-48ca-8218-3041fcce1836 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:33.280679 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1089d73-cb6d-48ca-8218-3041fcce1836 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:33.281138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1089d73-cb6d-48ca-8218-3041fcce1836 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:33.291896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1089d73-cb6d-48ca-8218-3041fcce1836 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:33.294707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1089d73-cb6d-48ca-8218-3041fcce1836 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-b1089d73-cb6d-48ca-8218-3041fcce1836 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:33.307057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:33.308084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:33.309105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:33.309105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:33.309105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:33.309105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:33.309105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:33.309105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:33.309105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:33.309105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:33.312860 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 271/538] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:33.320672 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1089d73-cb6d-48ca-8218-3041fcce1836 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:33.436942 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bcaa3c-c6bd-4c1d-bdca-f9fc04004f51 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:33.501473 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1089d73-cb6d-48ca-8218-3041fcce1836 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:33.505744 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a663ead-b63c-4600-b41a-553e359c0d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:33.506716 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a663ead-b63c-4600-b41a-553e359c0d9c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:33.507774 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a663ead-b63c-4600-b41a-553e359c0d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:33.508257 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a663ead-b63c-4600-b41a-553e359c0d9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:33.509260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a663ead-b63c-4600-b41a-553e359c0d9c None admin] RBAC: Adding query filter params (role.id=2b68b618a1b846f69bbe9aa08852a66f, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=95b665b9e27144b5bd684ece7ff465e2) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:33.509887 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a663ead-b63c-4600-b41a-553e359c0d9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:33.511545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a663ead-b63c-4600-b41a-553e359c0d9c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:33.524767 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 268/539] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:18:33 2022] GET /identity/v3/role_assignments?user.id=95b665b9e27144b5bd684ece7ff465e2&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=2b68b618a1b846f69bbe9aa08852a66f => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:33.584091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb2fe4-6d72-476b-908d-8eb0cdfa0ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:33.585175 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb2fe4-6d72-476b-908d-8eb0cdfa0ede None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:33.585643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb2fe4-6d72-476b-908d-8eb0cdfa0ede None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:33.586095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb2fe4-6d72-476b-908d-8eb0cdfa0ede None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:33.587039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb2fe4-6d72-476b-908d-8eb0cdfa0ede None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:33.587554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb2fe4-6d72-476b-908d-8eb0cdfa0ede None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:33.589231 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb2fe4-6d72-476b-908d-8eb0cdfa0ede None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:33.626228 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 272/540] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:33.632917 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a663ead-b63c-4600-b41a-553e359c0d9c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:33.724404 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:33.768247 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d44c1-1a9e-419b-874f-ee5077a0c029 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:33.769257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d44c1-1a9e-419b-874f-ee5077a0c029 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:33.769994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d44c1-1a9e-419b-874f-ee5077a0c029 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:33.771641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 273/541] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:18:33.777303 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bb792-fbb4-470b-97e2-104009df4485 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:33.777970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bb792-fbb4-470b-97e2-104009df4485 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:33.778817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bb792-fbb4-470b-97e2-104009df4485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:33.788522 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a663ead-b63c-4600-b41a-553e359c0d9c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:33.806042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-665bb792-fbb4-470b-97e2-104009df4485 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:33.807565 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-665bb792-fbb4-470b-97e2-104009df4485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:33.813076 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:33.831668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c306671-1700-4736-bece-e2d38063dc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:33.832311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c306671-1700-4736-bece-e2d38063dc28 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:33.832551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c306671-1700-4736-bece-e2d38063dc28 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:33.832819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c306671-1700-4736-bece-e2d38063dc28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:33.833539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c306671-1700-4736-bece-e2d38063dc28 None admin] RBAC: Adding query filter params (role.id=2b68b618a1b846f69bbe9aa08852a66f, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=aa65b6854dc54e289cc3e8e0b3746dd4) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:33.833840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c306671-1700-4736-bece-e2d38063dc28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:33.835361 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c306671-1700-4736-bece-e2d38063dc28 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:33.844734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 269/542] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:18:33 2022] GET /identity/v3/role_assignments?user.id=aa65b6854dc54e289cc3e8e0b3746dd4&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=2b68b618a1b846f69bbe9aa08852a66f => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:33.926055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 274/543] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:18:34.272359 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c306671-1700-4736-bece-e2d38063dc28 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:34.387946 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c306671-1700-4736-bece-e2d38063dc28 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:34.408018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:34.438782 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e2159-e456-4e84-8db7-491705ae96de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:34.439418 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e2159-e456-4e84-8db7-491705ae96de None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:34.439695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e2159-e456-4e84-8db7-491705ae96de None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:34.439963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e2159-e456-4e84-8db7-491705ae96de None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:34.455375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e2159-e456-4e84-8db7-491705ae96de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-063e2159-e456-4e84-8db7-491705ae96de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:34.460725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:34.461601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:34.462367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:34.462367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:34.462367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:34.462367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:34.462367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:34.462367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:34.462367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:34.462367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:34.462367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:34.462367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:34.462367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:34.462367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:34.462367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 270/544] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:34.468237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bb792-fbb4-470b-97e2-104009df4485 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:34.613997 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bb792-fbb4-470b-97e2-104009df4485 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:34.629896 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861ccef-a6c2-4a67-ad43-b0d7cbcefa71 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:34.630164 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861ccef-a6c2-4a67-ad43-b0d7cbcefa71 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:34.630445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861ccef-a6c2-4a67-ad43-b0d7cbcefa71 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:34.632099 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 271/545] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:18:34.637000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afe0c6-9ef8-4065-972f-0b5dd092f83a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:34.637283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afe0c6-9ef8-4065-972f-0b5dd092f83a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:34.637554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afe0c6-9ef8-4065-972f-0b5dd092f83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:34.651760 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c369b8-13c2-4479-9659-5100f5460f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:34.652291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c369b8-13c2-4479-9659-5100f5460f2e None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:34.652562 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c369b8-13c2-4479-9659-5100f5460f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:34.652826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c369b8-13c2-4479-9659-5100f5460f2e None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:34.653482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c369b8-13c2-4479-9659-5100f5460f2e None admin] RBAC: Adding query filter params (name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:34.653776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c369b8-13c2-4479-9659-5100f5460f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:34.655182 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c369b8-13c2-4479-9659-5100f5460f2e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:34.655637 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-94afe0c6-9ef8-4065-972f-0b5dd092f83a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:34.656823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-94afe0c6-9ef8-4065-972f-0b5dd092f83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:34.663576 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 275/546] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:34.669332 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c369b8-13c2-4479-9659-5100f5460f2e None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:34.672122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:34.748951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 272/547] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:18:34.830771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c369b8-13c2-4479-9659-5100f5460f2e None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:34.869956 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51616b-02ed-4fe5-8a1c-1000570f7b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:34.870981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51616b-02ed-4fe5-8a1c-1000570f7b19 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:34.871478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51616b-02ed-4fe5-8a1c-1000570f7b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:34.871937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51616b-02ed-4fe5-8a1c-1000570f7b19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:34.889522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51616b-02ed-4fe5-8a1c-1000570f7b19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:34.895372 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51616b-02ed-4fe5-8a1c-1000570f7b19 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-7c51616b-02ed-4fe5-8a1c-1000570f7b19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:34.910595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:34.911478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:34.912304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:34.912304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:34.912304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:34.914092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 276/548] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:34.920100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afe0c6-9ef8-4065-972f-0b5dd092f83a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:35.070364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afe0c6-9ef8-4065-972f-0b5dd092f83a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:35.072891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51616b-02ed-4fe5-8a1c-1000570f7b19 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:35.076823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:35.133874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa6c8b-0bfe-4f15-a5e5-c257049f6d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:35.134702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa6c8b-0bfe-4f15-a5e5-c257049f6d19 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:35.134981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa6c8b-0bfe-4f15-a5e5-c257049f6d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:35.135248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa6c8b-0bfe-4f15-a5e5-c257049f6d19 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:35.135893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa6c8b-0bfe-4f15-a5e5-c257049f6d19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:35.136195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa6c8b-0bfe-4f15-a5e5-c257049f6d19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:35.137591 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa6c8b-0bfe-4f15-a5e5-c257049f6d19 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:35.158182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 273/549] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:35.162564 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:35.168211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa6c8b-0bfe-4f15-a5e5-c257049f6d19 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:35.229547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51616b-02ed-4fe5-8a1c-1000570f7b19 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:35.265172 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead47291-6f55-41c5-8cf1-b1a5076f9af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:35.265857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47291-6f55-41c5-8cf1-b1a5076f9af3 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:35.266133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47291-6f55-41c5-8cf1-b1a5076f9af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:35.266404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47291-6f55-41c5-8cf1-b1a5076f9af3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:35.279801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead47291-6f55-41c5-8cf1-b1a5076f9af3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-ead47291-6f55-41c5-8cf1-b1a5076f9af3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:35.285822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:35.287540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:35.288444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:35.288444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:35.288444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:35.288444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:35.288444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:35.288444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:35.288444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:35.288444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:35.288444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:35.288444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:35.288444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:35.288444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:35.288444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 277/550] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:35.293967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead47291-6f55-41c5-8cf1-b1a5076f9af3 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:35.374821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa6c8b-0bfe-4f15-a5e5-c257049f6d19 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:35.424517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea04666-636e-452d-a9fe-49c25c638799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:35.425075 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea04666-636e-452d-a9fe-49c25c638799 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:35.425587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea04666-636e-452d-a9fe-49c25c638799 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:35.425853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea04666-636e-452d-a9fe-49c25c638799 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:35.435875 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea04666-636e-452d-a9fe-49c25c638799 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:35.437977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea04666-636e-452d-a9fe-49c25c638799 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-1ea04666-636e-452d-a9fe-49c25c638799 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:35.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:35.454694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:35.455761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:35.455761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:35.455761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:35.455761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:35.455761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:35.455761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:35.455761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:35.455761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:35.455761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 16:18:35.455761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:35.458593 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 274/551] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:18:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:35.459883 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52c1f9-b17b-4a9c-a517-cd6734da0e5b None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:35.460155 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52c1f9-b17b-4a9c-a517-cd6734da0e5b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:35.460424 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52c1f9-b17b-4a9c-a517-cd6734da0e5b None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:35.462785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 275/552] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:35.466868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52c1f9-b17b-4a9c-a517-cd6734da0e5b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:35.468156 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead47291-6f55-41c5-8cf1-b1a5076f9af3 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:35.485509 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:35.500148 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:35.510163 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd25df2-9cd0-4775-a3fc-1ab158a93ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:35.511118 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd25df2-9cd0-4775-a3fc-1ab158a93ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:35.511587 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd25df2-9cd0-4775-a3fc-1ab158a93ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:35.511951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd25df2-9cd0-4775-a3fc-1ab158a93ca7 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:35.512879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd25df2-9cd0-4775-a3fc-1ab158a93ca7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:35.513278 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd25df2-9cd0-4775-a3fc-1ab158a93ca7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:35.515553 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd25df2-9cd0-4775-a3fc-1ab158a93ca7 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:35.530384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 278/553] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:35.532139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d669ef-64dd-4f5c-84f8-408252f6a125 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:35.532521 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d669ef-64dd-4f5c-84f8-408252f6a125 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:35.532907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d669ef-64dd-4f5c-84f8-408252f6a125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:35.553559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-e3d669ef-64dd-4f5c-84f8-408252f6a125 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:35.554970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e3d669ef-64dd-4f5c-84f8-408252f6a125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:35.591295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52c1f9-b17b-4a9c-a517-cd6734da0e5b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:35.629700 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8698eb-9e9f-43a2-9618-fb4afb5336d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:35.630675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8698eb-9e9f-43a2-9618-fb4afb5336d2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:35.631050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8698eb-9e9f-43a2-9618-fb4afb5336d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:35.631469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8698eb-9e9f-43a2-9618-fb4afb5336d2 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:35.632485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8698eb-9e9f-43a2-9618-fb4afb5336d2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:35.632960 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8698eb-9e9f-43a2-9618-fb4afb5336d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:35.634800 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 279/554] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:35.635775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d669ef-64dd-4f5c-84f8-408252f6a125 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:35.638731 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8698eb-9e9f-43a2-9618-fb4afb5336d2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:35.675422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 276/555] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Sep 9 16:18:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:35.687921 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8698eb-9e9f-43a2-9618-fb4afb5336d2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:35.749648 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d669ef-64dd-4f5c-84f8-408252f6a125 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:35.758605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:35.787991 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f699d-129b-4da7-871e-7fc50b7097ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:35.788917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f699d-129b-4da7-871e-7fc50b7097ce None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:35.789641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f699d-129b-4da7-871e-7fc50b7097ce None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:35.789914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f699d-129b-4da7-871e-7fc50b7097ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:35.801891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f699d-129b-4da7-871e-7fc50b7097ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:35.803415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f699d-129b-4da7-871e-7fc50b7097ce None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:35.813347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-834f699d-129b-4da7-871e-7fc50b7097ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:35.820547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:35.822826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:35.824740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:35.824740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:35.824740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:35.824740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 280/556] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:35.829449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f699d-129b-4da7-871e-7fc50b7097ce None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:35.848261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8698eb-9e9f-43a2-9618-fb4afb5336d2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:35.908158 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c636d-19a3-4da6-a1ff-795fb5ecee17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:35.908769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c636d-19a3-4da6-a1ff-795fb5ecee17 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:35.909047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c636d-19a3-4da6-a1ff-795fb5ecee17 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:35.909312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c636d-19a3-4da6-a1ff-795fb5ecee17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:36.016014 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f699d-129b-4da7-871e-7fc50b7097ce None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:36.040221 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c636d-19a3-4da6-a1ff-795fb5ecee17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:36.041695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c636d-19a3-4da6-a1ff-795fb5ecee17 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:36.069475 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0c518-8bae-45b4-9039-f3bde94b0435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:36.070092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0c518-8bae-45b4-9039-f3bde94b0435 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:36.070738 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0c518-8bae-45b4-9039-f3bde94b0435 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:36.070738 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0c518-8bae-45b4-9039-f3bde94b0435 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:36.071368 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0c518-8bae-45b4-9039-f3bde94b0435 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:36.071618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0c518-8bae-45b4-9039-f3bde94b0435 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:36.076852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0c518-8bae-45b4-9039-f3bde94b0435 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:36.088664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 281/557] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:36.089732 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0c518-8bae-45b4-9039-f3bde94b0435 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:36.111423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:36.133187 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:36.156924 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 277/558] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 16:18:36.158331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20116ec4-66f2-424b-9f86-87a7e370f939 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:36.158907 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20116ec4-66f2-424b-9f86-87a7e370f939 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:36.158907 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20116ec4-66f2-424b-9f86-87a7e370f939 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:36.160311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 278/559] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:36.160914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20116ec4-66f2-424b-9f86-87a7e370f939 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:36.201735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0c518-8bae-45b4-9039-f3bde94b0435 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:36.237693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed7d0e-ec7e-452a-a337-c6aa53633c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:36.238444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed7d0e-ec7e-452a-a337-c6aa53633c1b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:36.238746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed7d0e-ec7e-452a-a337-c6aa53633c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:36.239014 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed7d0e-ec7e-452a-a337-c6aa53633c1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:36.250455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed7d0e-ec7e-452a-a337-c6aa53633c1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-65ed7d0e-ec7e-452a-a337-c6aa53633c1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:36.256474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:36.257471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:36.258299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:36.258299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:36.258299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:36.258299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:36.258299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:36.258299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:36.258299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:36.258299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:36.258299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:36.258299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:36.258299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:36.258299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:36.258299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 282/560] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:36.260001 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed7d0e-ec7e-452a-a337-c6aa53633c1b None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:36.267121 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20116ec4-66f2-424b-9f86-87a7e370f939 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:36.302714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651bac6-952d-455c-8754-ee8235134ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:36.303992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651bac6-952d-455c-8754-ee8235134ccc None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:36.304755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651bac6-952d-455c-8754-ee8235134ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:36.305216 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651bac6-952d-455c-8754-ee8235134ccc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:36.312859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651bac6-952d-455c-8754-ee8235134ccc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:36.314590 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651bac6-952d-455c-8754-ee8235134ccc None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-8651bac6-952d-455c-8754-ee8235134ccc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:36.326926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:36.328125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:36.329283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:36.329283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:36.329283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:36.329283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:36.329283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:36.329283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:36.329283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:36.329283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:36.329283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:18:36.329283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:36.332834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 279/561] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Sep 9 16:18:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:36.334045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a0366-fd37-40f8-80c8-da635511197c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:36.334351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a0366-fd37-40f8-80c8-da635511197c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:36.334652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a0366-fd37-40f8-80c8-da635511197c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:36.343951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:36.353034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-9b8a0366-fd37-40f8-80c8-da635511197c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:36.354229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9b8a0366-fd37-40f8-80c8-da635511197c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:36.367519 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed7d0e-ec7e-452a-a337-c6aa53633c1b None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:36.392348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:36.404543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d2c4f-9928-43af-8cc8-f5a59d027c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:36.405255 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d2c4f-9928-43af-8cc8-f5a59d027c72 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:36.405596 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d2c4f-9928-43af-8cc8-f5a59d027c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:36.405933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d2c4f-9928-43af-8cc8-f5a59d027c72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:36.416649 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d2c4f-9928-43af-8cc8-f5a59d027c72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:36.418185 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d2c4f-9928-43af-8cc8-f5a59d027c72 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:36.425256 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 280/562] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:36.425886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a0366-fd37-40f8-80c8-da635511197c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-fd4d2c4f-9928-43af-8cc8-f5a59d027c72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:36.443699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:36.444908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:36.445956 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:36.445956 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:36.445956 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:36.452990 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 283/563] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:36.454417 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d2c4f-9928-43af-8cc8-f5a59d027c72 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:36.537603 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a0366-fd37-40f8-80c8-da635511197c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:36.572936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b29585-7210-49cd-9228-0e03988237f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:36.573646 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b29585-7210-49cd-9228-0e03988237f5 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:36.573990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b29585-7210-49cd-9228-0e03988237f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:36.574320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b29585-7210-49cd-9228-0e03988237f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:36.575083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b29585-7210-49cd-9228-0e03988237f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:36.575451 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b29585-7210-49cd-9228-0e03988237f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:36.576894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b29585-7210-49cd-9228-0e03988237f5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:36.589823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 281/564] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:36.589823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b29585-7210-49cd-9228-0e03988237f5 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:36.603237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d2c4f-9928-43af-8cc8-f5a59d027c72 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:36.623602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:36.631871 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:36.638379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8dd71-835b-4169-ae81-a1342ff8e3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:36.639084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8dd71-835b-4169-ae81-a1342ff8e3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:36.639427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8dd71-835b-4169-ae81-a1342ff8e3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:36.639761 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8dd71-835b-4169-ae81-a1342ff8e3e6 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:36.640490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8dd71-835b-4169-ae81-a1342ff8e3e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:36.640790 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8dd71-835b-4169-ae81-a1342ff8e3e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:36.642220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8dd71-835b-4169-ae81-a1342ff8e3e6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:36.659790 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 284/565] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Sep 9 16:18:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:36.661152 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe7930-9055-4ece-8840-f83f5978a3ab None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:36.661430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe7930-9055-4ece-8840-f83f5978a3ab None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:36.661698 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe7930-9055-4ece-8840-f83f5978a3ab None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:36.663084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 285/566] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:36.663680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe7930-9055-4ece-8840-f83f5978a3ab None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:36.704554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b29585-7210-49cd-9228-0e03988237f5 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:36.737776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00534ce-2a82-4345-8d36-45489f52ffa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:36.738381 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00534ce-2a82-4345-8d36-45489f52ffa0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:36.738695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00534ce-2a82-4345-8d36-45489f52ffa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:36.738962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00534ce-2a82-4345-8d36-45489f52ffa0 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:36.739594 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00534ce-2a82-4345-8d36-45489f52ffa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:36.739891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00534ce-2a82-4345-8d36-45489f52ffa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:36.741757 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00534ce-2a82-4345-8d36-45489f52ffa0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:36.762529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 282/567] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:36.762529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00534ce-2a82-4345-8d36-45489f52ffa0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:36.765607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe7930-9055-4ece-8840-f83f5978a3ab None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:36.802056 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703eab8e-87ff-48f6-9240-134564cc7953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:36.802613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703eab8e-87ff-48f6-9240-134564cc7953 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:36.802891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703eab8e-87ff-48f6-9240-134564cc7953 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:36.803257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703eab8e-87ff-48f6-9240-134564cc7953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:36.813468 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703eab8e-87ff-48f6-9240-134564cc7953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:36.814954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703eab8e-87ff-48f6-9240-134564cc7953 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-703eab8e-87ff-48f6-9240-134564cc7953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:36.829814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:36.830791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:36.831673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:36.831673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:36.831673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:36.832128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 286/568] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:36.833397 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f9d80-f6e9-4e5d-843c-10cecf97b2f0 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:36.833676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f9d80-f6e9-4e5d-843c-10cecf97b2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:36.833930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f9d80-f6e9-4e5d-843c-10cecf97b2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:36.851806 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-7e7f9d80-f6e9-4e5d-843c-10cecf97b2f0 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:36.852989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7e7f9d80-f6e9-4e5d-843c-10cecf97b2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:36.857841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:36.864742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00534ce-2a82-4345-8d36-45489f52ffa0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:36.885791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:36.897346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd535f84-42fd-486d-b701-12300fde08c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:36.897974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd535f84-42fd-486d-b701-12300fde08c2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:36.898248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd535f84-42fd-486d-b701-12300fde08c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:36.898541 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd535f84-42fd-486d-b701-12300fde08c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:36.909495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd535f84-42fd-486d-b701-12300fde08c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:36.910987 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd535f84-42fd-486d-b701-12300fde08c2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:36.919200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 287/569] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:36.919996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f9d80-f6e9-4e5d-843c-10cecf97b2f0 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-dd535f84-42fd-486d-b701-12300fde08c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:36.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:36.926370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:36.927575 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:36.927575 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:36.927575 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:36.927575 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 283/570] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:36.927803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd535f84-42fd-486d-b701-12300fde08c2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:37.028742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd535f84-42fd-486d-b701-12300fde08c2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:37.029216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f9d80-f6e9-4e5d-843c-10cecf97b2f0 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:37.064669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e7220-2a87-497a-81d0-046df9774274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:37.065173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c212b40-b4db-4345-801d-a56dd210fe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:37.065278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e7220-2a87-497a-81d0-046df9774274 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:37.065355 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c212b40-b4db-4345-801d-a56dd210fe64 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:37.065589 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c212b40-b4db-4345-801d-a56dd210fe64 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:37.065685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e7220-2a87-497a-81d0-046df9774274 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:37.065768 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c212b40-b4db-4345-801d-a56dd210fe64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:37.065839 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e7220-2a87-497a-81d0-046df9774274 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:37.074653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e7220-2a87-497a-81d0-046df9774274 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:37.074653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e7220-2a87-497a-81d0-046df9774274 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:37.078935 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c212b40-b4db-4345-801d-a56dd210fe64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-0c212b40-b4db-4345-801d-a56dd210fe64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:37.085643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:37.086875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:37.087701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:37.087701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:37.087701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:37.087701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:37.087701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:37.087701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:37.087701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:37.087701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:37.087701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:37.087701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:37.087701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:37.087701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:37.087701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 288/571] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:37.087701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c212b40-b4db-4345-801d-a56dd210fe64 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-f76e7220-2a87-497a-81d0-046df9774274 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:37.090847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:37.091797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:37.093151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:37.093151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:37.093151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:37.093151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:37.093151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:37.093151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:37.093151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:37.093151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:37.093151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 284/572] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:37.093151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e7220-2a87-497a-81d0-046df9774274 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:37.105527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:37.123710 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:37.198254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c212b40-b4db-4345-801d-a56dd210fe64 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:37.202733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e7220-2a87-497a-81d0-046df9774274 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:37.231996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01589689-a5d7-48af-80f5-61b9f1148f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:37.232608 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01589689-a5d7-48af-80f5-61b9f1148f0f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:37.232882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01589689-a5d7-48af-80f5-61b9f1148f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:37.233151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01589689-a5d7-48af-80f5-61b9f1148f0f None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:37.233788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01589689-a5d7-48af-80f5-61b9f1148f0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:37.234087 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01589689-a5d7-48af-80f5-61b9f1148f0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:37.234539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6892c-afe2-48a6-9dc9-5a183de9843a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:37.235135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6892c-afe2-48a6-9dc9-5a183de9843a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:37.235410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6892c-afe2-48a6-9dc9-5a183de9843a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:37.235502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01589689-a5d7-48af-80f5-61b9f1148f0f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:37.235683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6892c-afe2-48a6-9dc9-5a183de9843a None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:37.236309 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6892c-afe2-48a6-9dc9-5a183de9843a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:37.236605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6892c-afe2-48a6-9dc9-5a183de9843a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:37.237955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6892c-afe2-48a6-9dc9-5a183de9843a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:37.246760 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 289/573] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:37.247544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01589689-a5d7-48af-80f5-61b9f1148f0f None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:37.249618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 285/574] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:37.250659 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6892c-afe2-48a6-9dc9-5a183de9843a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:37.314882 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:37.337176 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:37.347215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01589689-a5d7-48af-80f5-61b9f1148f0f None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:37.349437 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6892c-afe2-48a6-9dc9-5a183de9843a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:37.380229 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6465f3-9c9f-451b-be3b-aa9622f68837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:37.380834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6465f3-9c9f-451b-be3b-aa9622f68837 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:37.381127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6465f3-9c9f-451b-be3b-aa9622f68837 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:37.381563 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6465f3-9c9f-451b-be3b-aa9622f68837 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:37.382218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6465f3-9c9f-451b-be3b-aa9622f68837 None admin] RBAC: Adding query filter params (name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:37.382533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6465f3-9c9f-451b-be3b-aa9622f68837 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:37.383991 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6465f3-9c9f-451b-be3b-aa9622f68837 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:37.384460 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240546bf-e639-4e6f-ab0c-a188aeddc081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:37.385134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240546bf-e639-4e6f-ab0c-a188aeddc081 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:37.385639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240546bf-e639-4e6f-ab0c-a188aeddc081 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:37.385983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240546bf-e639-4e6f-ab0c-a188aeddc081 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:37.386764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240546bf-e639-4e6f-ab0c-a188aeddc081 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:37.387066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240546bf-e639-4e6f-ab0c-a188aeddc081 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:37.388542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240546bf-e639-4e6f-ab0c-a188aeddc081 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:37.392169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 290/575] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:37.392974 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6465f3-9c9f-451b-be3b-aa9622f68837 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:37.400698 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 286/576] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:37.401700 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240546bf-e639-4e6f-ab0c-a188aeddc081 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:37.493696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6465f3-9c9f-451b-be3b-aa9622f68837 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:37.501206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240546bf-e639-4e6f-ab0c-a188aeddc081 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:37.526636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:37.529700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b8e34-e372-4304-9b8d-8e3ac5145f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:37.530546 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b8e34-e372-4304-9b8d-8e3ac5145f0c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:37.531057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b8e34-e372-4304-9b8d-8e3ac5145f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:37.531240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b8e34-e372-4304-9b8d-8e3ac5145f0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:37.535417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12930469-0708-48b6-b37b-75473725fbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:37.536026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12930469-0708-48b6-b37b-75473725fbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:37.536301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12930469-0708-48b6-b37b-75473725fbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:37.536564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12930469-0708-48b6-b37b-75473725fbd1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:37.541580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b8e34-e372-4304-9b8d-8e3ac5145f0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:37.543977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12930469-0708-48b6-b37b-75473725fbd1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:37.545435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12930469-0708-48b6-b37b-75473725fbd1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-603b8e34-e372-4304-9b8d-8e3ac5145f0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:37.546851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:37.547920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:37.548864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:37.548864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:37.548864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:37.548864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:37.548864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:37.548864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:37.548864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:37.548864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:37.548864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:37.548864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:37.548864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:37.548864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:37.548864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 291/577] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:37.548864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b8e34-e372-4304-9b8d-8e3ac5145f0c None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-12930469-0708-48b6-b37b-75473725fbd1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:37.558727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:37.559853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:37.561330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:37.561330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:37.561330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:37.561330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:37.561330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:37.561330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:37.561330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:37.561330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:37.561330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 16:18:37.561330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:37.561330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 287/578] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:37.561330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12930469-0708-48b6-b37b-75473725fbd1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:37.570391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:37.653421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b8e34-e372-4304-9b8d-8e3ac5145f0c None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:37.658857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12930469-0708-48b6-b37b-75473725fbd1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:37.687167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25247212-9b51-4741-80d2-7ccc23189590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:37.687796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25247212-9b51-4741-80d2-7ccc23189590 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:37.688068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25247212-9b51-4741-80d2-7ccc23189590 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:37.688342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25247212-9b51-4741-80d2-7ccc23189590 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:37.692686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c6132-1e28-4e0c-b40d-f12b3b3da5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:37.693301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c6132-1e28-4e0c-b40d-f12b3b3da5ab None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:37.693574 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c6132-1e28-4e0c-b40d-f12b3b3da5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:37.693873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c6132-1e28-4e0c-b40d-f12b3b3da5ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:37.695312 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25247212-9b51-4741-80d2-7ccc23189590 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:37.696743 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25247212-9b51-4741-80d2-7ccc23189590 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:37.704250 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c6132-1e28-4e0c-b40d-f12b3b3da5ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:37.705677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c6132-1e28-4e0c-b40d-f12b3b3da5ab None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-25247212-9b51-4741-80d2-7ccc23189590 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:37.707711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:37.708630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:37.714379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:37.714379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:37.714379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:37.714379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:37.714379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:37.714379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:37.714379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:37.714379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:37.714379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 292/579] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:37.714379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25247212-9b51-4741-80d2-7ccc23189590 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-784c6132-1e28-4e0c-b40d-f12b3b3da5ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:37.720797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:37.721886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:37.723240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:37.723240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:37.723240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:37.723240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 288/580] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:37.724477 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c6132-1e28-4e0c-b40d-f12b3b3da5ab None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:37.763733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:37.778908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:37.808405 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25247212-9b51-4741-80d2-7ccc23189590 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:37.824074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c6132-1e28-4e0c-b40d-f12b3b3da5ab None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:37.841712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed67a3a-3966-4fa4-8bd7-797e8fdcfa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:37.842310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed67a3a-3966-4fa4-8bd7-797e8fdcfa5b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:37.842615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed67a3a-3966-4fa4-8bd7-797e8fdcfa5b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:37.842885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed67a3a-3966-4fa4-8bd7-797e8fdcfa5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:37.843607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed67a3a-3966-4fa4-8bd7-797e8fdcfa5b None admin] RBAC: Adding query filter params (role.id=2b68b618a1b846f69bbe9aa08852a66f, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=e926034945b44087b378b3468954a18a) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:37.843907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed67a3a-3966-4fa4-8bd7-797e8fdcfa5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:37.845341 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed67a3a-3966-4fa4-8bd7-797e8fdcfa5b None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:37.850636 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 293/581] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:18:37 2022] GET /identity/v3/role_assignments?user.id=e926034945b44087b378b3468954a18a&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=2b68b618a1b846f69bbe9aa08852a66f => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:37.851374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed67a3a-3966-4fa4-8bd7-797e8fdcfa5b None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:37.858239 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e86d3-cc21-42a9-9023-516ea3782a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:37.858902 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e86d3-cc21-42a9-9023-516ea3782a4b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:37.859179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e86d3-cc21-42a9-9023-516ea3782a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:37.859444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e86d3-cc21-42a9-9023-516ea3782a4b None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:37.860101 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e86d3-cc21-42a9-9023-516ea3782a4b None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:37.860411 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e86d3-cc21-42a9-9023-516ea3782a4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:37.861850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e86d3-cc21-42a9-9023-516ea3782a4b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:37.871469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 289/582] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:37.872353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e86d3-cc21-42a9-9023-516ea3782a4b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:37.973902 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed67a3a-3966-4fa4-8bd7-797e8fdcfa5b None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:37.981927 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:37.985619 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e86d3-cc21-42a9-9023-516ea3782a4b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:38.011600 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a073355-4c66-4076-b227-fde891617552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:38.012494 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a073355-4c66-4076-b227-fde891617552 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:38.012917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a073355-4c66-4076-b227-fde891617552 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:38.013360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a073355-4c66-4076-b227-fde891617552 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:38.014254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a073355-4c66-4076-b227-fde891617552 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:38.014590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a073355-4c66-4076-b227-fde891617552 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:38.016052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a073355-4c66-4076-b227-fde891617552 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:38.019721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:38.023536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b7855-2a8c-47ec-8abe-fc80c1e04961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:38.024151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b7855-2a8c-47ec-8abe-fc80c1e04961 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:38.024427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b7855-2a8c-47ec-8abe-fc80c1e04961 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:38.024692 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b7855-2a8c-47ec-8abe-fc80c1e04961 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:38.025387 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b7855-2a8c-47ec-8abe-fc80c1e04961 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:38.025687 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b7855-2a8c-47ec-8abe-fc80c1e04961 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:38.027173 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b7855-2a8c-47ec-8abe-fc80c1e04961 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:38.041374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 290/583] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:38.042290 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b7855-2a8c-47ec-8abe-fc80c1e04961 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:38.047928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 294/584] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Sep 9 16:18:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:38.048888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a073355-4c66-4076-b227-fde891617552 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:38.149513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b7855-2a8c-47ec-8abe-fc80c1e04961 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:38.161612 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a073355-4c66-4076-b227-fde891617552 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:38.184444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39741f06-cb11-4ddb-a4f5-e80a6791e504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:38.185054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39741f06-cb11-4ddb-a4f5-e80a6791e504 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:38.185488 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39741f06-cb11-4ddb-a4f5-e80a6791e504 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:38.185761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39741f06-cb11-4ddb-a4f5-e80a6791e504 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:38.186405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39741f06-cb11-4ddb-a4f5-e80a6791e504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:38.186732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39741f06-cb11-4ddb-a4f5-e80a6791e504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:38.188102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39741f06-cb11-4ddb-a4f5-e80a6791e504 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:38.196978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46210d-cf24-4275-8518-c149f5d78060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:38.197605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46210d-cf24-4275-8518-c149f5d78060 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:38.197879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46210d-cf24-4275-8518-c149f5d78060 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:38.198155 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46210d-cf24-4275-8518-c149f5d78060 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:38.204739 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 291/585] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:38.204739 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39741f06-cb11-4ddb-a4f5-e80a6791e504 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:38.213585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46210d-cf24-4275-8518-c149f5d78060 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:38.215081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46210d-cf24-4275-8518-c149f5d78060 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:38.224100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-ba46210d-cf24-4275-8518-c149f5d78060 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:38.230923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:38.231855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:38.232742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:38.232742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:38.232742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:38.232742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 295/586] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:38.232961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46210d-cf24-4275-8518-c149f5d78060 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:38.278946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:38.305444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39741f06-cb11-4ddb-a4f5-e80a6791e504 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:38.338588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46210d-cf24-4275-8518-c149f5d78060 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:38.338815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6d18d-6818-45b0-a1ee-4ee1efb9dd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:38.339281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6d18d-6818-45b0-a1ee-4ee1efb9dd5f None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:38.339555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6d18d-6818-45b0-a1ee-4ee1efb9dd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:38.339824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6d18d-6818-45b0-a1ee-4ee1efb9dd5f None admin] PATH_INFO: `/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/152a41ad598b4b01b1881d928c53c848/roles/2b68b618a1b846f69bbe9aa08852a66f` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:38.354092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6d18d-6818-45b0-a1ee-4ee1efb9dd5f None admin] RBAC: Authorizing `identity:create_grant(project_id=253ff30a56444df28d44c4b7eaadb276, user_id=152a41ad598b4b01b1881d928c53c848, role_id=2b68b618a1b846f69bbe9aa08852a66f)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:38.355564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6d18d-6818-45b0-a1ee-4ee1efb9dd5f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:38.378537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 292/587] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:18:38 2022] PUT /identity/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/152a41ad598b4b01b1881d928c53c848/roles/2b68b618a1b846f69bbe9aa08852a66f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:18:38.378537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6d18d-6818-45b0-a1ee-4ee1efb9dd5f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:38.387132 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd3eb9-f8a0-4ea1-bc29-ecb671445707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:38.387891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd3eb9-f8a0-4ea1-bc29-ecb671445707 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:38.388426 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd3eb9-f8a0-4ea1-bc29-ecb671445707 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:38.388975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd3eb9-f8a0-4ea1-bc29-ecb671445707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:38.402734 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd3eb9-f8a0-4ea1-bc29-ecb671445707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:38.404224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd3eb9-f8a0-4ea1-bc29-ecb671445707 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-3fdd3eb9-f8a0-4ea1-bc29-ecb671445707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:38.422135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:38.423401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:38.424356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:38.424356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:38.424356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:38.424356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 296/588] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:38.424688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd3eb9-f8a0-4ea1-bc29-ecb671445707 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:38.458659 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:38.496577 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6d18d-6818-45b0-a1ee-4ee1efb9dd5f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:38.535840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91cfd2b-bbbd-4c26-a308-9226cc27a1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:38.536492 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cfd2b-bbbd-4c26-a308-9226cc27a1de None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:38.536815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cfd2b-bbbd-4c26-a308-9226cc27a1de None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:38.537091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cfd2b-bbbd-4c26-a308-9226cc27a1de None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:38.545723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91cfd2b-bbbd-4c26-a308-9226cc27a1de None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:38.547390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91cfd2b-bbbd-4c26-a308-9226cc27a1de None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c91cfd2b-bbbd-4c26-a308-9226cc27a1de None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:38.560874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:38.561898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:38.562984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:38.562984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:38.562984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:38.562984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:38.562984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:38.562984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:38.562984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:38.562984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:38.562984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 16:18:38.562984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:38.562984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 293/589] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:38.563567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91cfd2b-bbbd-4c26-a308-9226cc27a1de None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:38.568091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:38.572189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd3eb9-f8a0-4ea1-bc29-ecb671445707 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:38.612709 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e638c56-6e00-4b2c-b92d-cd2f538deb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:38.613265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e638c56-6e00-4b2c-b92d-cd2f538deb56 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:38.613538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e638c56-6e00-4b2c-b92d-cd2f538deb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:38.615731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e638c56-6e00-4b2c-b92d-cd2f538deb56 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:38.615731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e638c56-6e00-4b2c-b92d-cd2f538deb56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:38.615731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e638c56-6e00-4b2c-b92d-cd2f538deb56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:38.617214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e638c56-6e00-4b2c-b92d-cd2f538deb56 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:38.640007 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 297/590] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:38.640927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e638c56-6e00-4b2c-b92d-cd2f538deb56 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:38.686398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91cfd2b-bbbd-4c26-a308-9226cc27a1de None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:38.716766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:38.724998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675be126-f9a9-4eea-94d8-d9c1b0c8b09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:38.725452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675be126-f9a9-4eea-94d8-d9c1b0c8b09d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:38.725728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675be126-f9a9-4eea-94d8-d9c1b0c8b09d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:38.725996 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675be126-f9a9-4eea-94d8-d9c1b0c8b09d None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:38.726780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675be126-f9a9-4eea-94d8-d9c1b0c8b09d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:38.726977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675be126-f9a9-4eea-94d8-d9c1b0c8b09d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:38.728559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675be126-f9a9-4eea-94d8-d9c1b0c8b09d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:38.742210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 294/591] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:38.743091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675be126-f9a9-4eea-94d8-d9c1b0c8b09d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:38.777210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e638c56-6e00-4b2c-b92d-cd2f538deb56 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:38.815030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ded4eb-e26c-43d0-bcb2-c66817f3f382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:38.815691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ded4eb-e26c-43d0-bcb2-c66817f3f382 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:38.815905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ded4eb-e26c-43d0-bcb2-c66817f3f382 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:38.816528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ded4eb-e26c-43d0-bcb2-c66817f3f382 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:38.817253 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ded4eb-e26c-43d0-bcb2-c66817f3f382 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:38.817550 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ded4eb-e26c-43d0-bcb2-c66817f3f382 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:38.819078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ded4eb-e26c-43d0-bcb2-c66817f3f382 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:38.837875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 298/592] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Sep 9 16:18:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:38.842819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ded4eb-e26c-43d0-bcb2-c66817f3f382 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:38.861337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:38.882653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675be126-f9a9-4eea-94d8-d9c1b0c8b09d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:38.924970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbcaed7-bc9e-4705-9cdf-8ea31831672d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:38.925604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbcaed7-bc9e-4705-9cdf-8ea31831672d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:38.925876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbcaed7-bc9e-4705-9cdf-8ea31831672d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:38.926231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbcaed7-bc9e-4705-9cdf-8ea31831672d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:38.933835 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbcaed7-bc9e-4705-9cdf-8ea31831672d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:38.935326 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbcaed7-bc9e-4705-9cdf-8ea31831672d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-bbbcaed7-bc9e-4705-9cdf-8ea31831672d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:38.949601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:38.950618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:38.951515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:38.951515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:38.951515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:38.951515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:38.951515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:38.951515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:38.951515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:38.951515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:38.951515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 16:18:38.951515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:38.951515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 295/593] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:38.951515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbcaed7-bc9e-4705-9cdf-8ea31831672d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:38.976714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ded4eb-e26c-43d0-bcb2-c66817f3f382 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:39.002385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:39.012199 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cdb0c-9564-4518-a9a4-5617274e639f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:39.013149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cdb0c-9564-4518-a9a4-5617274e639f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:39.013648 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cdb0c-9564-4518-a9a4-5617274e639f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:39.014112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cdb0c-9564-4518-a9a4-5617274e639f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:39.021298 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cdb0c-9564-4518-a9a4-5617274e639f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:39.023071 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cdb0c-9564-4518-a9a4-5617274e639f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-9a7cdb0c-9564-4518-a9a4-5617274e639f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:39.034764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:39.035662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:39.036540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:39.036540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:39.036540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:39.036540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:39.036540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:39.036540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:39.036540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:39.036540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:39.043620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 299/594] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:39.044716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cdb0c-9564-4518-a9a4-5617274e639f None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:39.056365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbcaed7-bc9e-4705-9cdf-8ea31831672d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:39.097516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2483412-ebf3-4651-a417-bf1daf5dadd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:39.098294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2483412-ebf3-4651-a417-bf1daf5dadd7 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:39.098595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2483412-ebf3-4651-a417-bf1daf5dadd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:39.098866 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2483412-ebf3-4651-a417-bf1daf5dadd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:39.109891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2483412-ebf3-4651-a417-bf1daf5dadd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:39.111379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2483412-ebf3-4651-a417-bf1daf5dadd7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:39.125289 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-f2483412-ebf3-4651-a417-bf1daf5dadd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:39.126159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:39.127104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:39.127982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:39.127982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:39.127982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:39.132387 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 296/595] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:39.133626 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2483412-ebf3-4651-a417-bf1daf5dadd7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:39.149739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cdb0c-9564-4518-a9a4-5617274e639f None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:39.184606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7be66-4257-4025-9ee7-08dd8f8be395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:39.185226 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7be66-4257-4025-9ee7-08dd8f8be395 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:39.185532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7be66-4257-4025-9ee7-08dd8f8be395 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:39.185772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7be66-4257-4025-9ee7-08dd8f8be395 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:39.186523 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7be66-4257-4025-9ee7-08dd8f8be395 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:39.186830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7be66-4257-4025-9ee7-08dd8f8be395 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:39.188427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7be66-4257-4025-9ee7-08dd8f8be395 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:39.213824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 300/596] 10.222.0.23 () {56 vars in 1139 bytes} [Fri Sep 9 16:18:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:39.214643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7be66-4257-4025-9ee7-08dd8f8be395 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:39.244422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2483412-ebf3-4651-a417-bf1daf5dadd7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:39.246051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:39.285823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d88d6-0afd-46bc-9602-74bb2406e742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:39.286425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d88d6-0afd-46bc-9602-74bb2406e742 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:39.286742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d88d6-0afd-46bc-9602-74bb2406e742 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:39.287007 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d88d6-0afd-46bc-9602-74bb2406e742 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:39.287762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d88d6-0afd-46bc-9602-74bb2406e742 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:39.288156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d88d6-0afd-46bc-9602-74bb2406e742 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:39.289679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d88d6-0afd-46bc-9602-74bb2406e742 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:39.302312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 297/597] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:39.303191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d88d6-0afd-46bc-9602-74bb2406e742 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:39.324259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7be66-4257-4025-9ee7-08dd8f8be395 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:39.362614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25c91d-612f-4950-8d75-2fbe5b230083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:39.363238 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25c91d-612f-4950-8d75-2fbe5b230083 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:39.363511 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25c91d-612f-4950-8d75-2fbe5b230083 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:39.363977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25c91d-612f-4950-8d75-2fbe5b230083 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:39.365067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25c91d-612f-4950-8d75-2fbe5b230083 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:39.365384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25c91d-612f-4950-8d75-2fbe5b230083 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:39.367195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25c91d-612f-4950-8d75-2fbe5b230083 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:39.369156 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:39.387081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 301/598] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:39.388340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25c91d-612f-4950-8d75-2fbe5b230083 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:39.437294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d88d6-0afd-46bc-9602-74bb2406e742 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:39.476696 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb588c1-78c7-4bbe-8dec-6382341d0c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:39.477422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb588c1-78c7-4bbe-8dec-6382341d0c22 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:39.477765 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb588c1-78c7-4bbe-8dec-6382341d0c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:39.478155 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb588c1-78c7-4bbe-8dec-6382341d0c22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:39.489199 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb588c1-78c7-4bbe-8dec-6382341d0c22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:39.490806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb588c1-78c7-4bbe-8dec-6382341d0c22 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:39.502403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25c91d-612f-4950-8d75-2fbe5b230083 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-3bb588c1-78c7-4bbe-8dec-6382341d0c22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:39.505551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:39.506750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:39.507875 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:39.507875 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:39.507875 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:39.513211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 298/599] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:39.514113 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb588c1-78c7-4bbe-8dec-6382341d0c22 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:39.531251 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:39.547593 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27180112-02d9-4ca3-93a7-1acd73ee860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:39.548502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27180112-02d9-4ca3-93a7-1acd73ee860a None admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:39.548917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27180112-02d9-4ca3-93a7-1acd73ee860a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:39.549484 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27180112-02d9-4ca3-93a7-1acd73ee860a None admin] PATH_INFO: `/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/62a2fb70c9ae4f4b81ac4de513b868cc/roles/2b68b618a1b846f69bbe9aa08852a66f` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:39.567039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27180112-02d9-4ca3-93a7-1acd73ee860a None admin] RBAC: Authorizing `identity:create_grant(project_id=253ff30a56444df28d44c4b7eaadb276, user_id=62a2fb70c9ae4f4b81ac4de513b868cc, role_id=2b68b618a1b846f69bbe9aa08852a66f)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:39.569295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27180112-02d9-4ca3-93a7-1acd73ee860a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:39.587139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 302/600] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:18:39 2022] PUT /identity/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/62a2fb70c9ae4f4b81ac4de513b868cc/roles/2b68b618a1b846f69bbe9aa08852a66f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:18:39.588185 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27180112-02d9-4ca3-93a7-1acd73ee860a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:39.669020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb588c1-78c7-4bbe-8dec-6382341d0c22 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:39.696065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:39.717681 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204b76a-975f-48a4-b710-0920e9cbb813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:39.718783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204b76a-975f-48a4-b710-0920e9cbb813 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:39.719255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204b76a-975f-48a4-b710-0920e9cbb813 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:39.719714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204b76a-975f-48a4-b710-0920e9cbb813 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:39.729470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204b76a-975f-48a4-b710-0920e9cbb813 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:39.731715 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204b76a-975f-48a4-b710-0920e9cbb813 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-a204b76a-975f-48a4-b710-0920e9cbb813 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:39.746595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:39.747532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:39.748401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:39.748401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:39.748401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:39.748401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:39.748401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:39.748401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:39.748401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:39.748401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:39.751553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 299/601] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:39.758807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204b76a-975f-48a4-b710-0920e9cbb813 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:39.823617 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27180112-02d9-4ca3-93a7-1acd73ee860a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:39.851395 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:39.884137 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204b76a-975f-48a4-b710-0920e9cbb813 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:39.890365 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e6a34-477a-4152-a9bd-8b36dd0e9446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:39.891037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e6a34-477a-4152-a9bd-8b36dd0e9446 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:39.891313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e6a34-477a-4152-a9bd-8b36dd0e9446 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:39.891580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e6a34-477a-4152-a9bd-8b36dd0e9446 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:39.892307 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e6a34-477a-4152-a9bd-8b36dd0e9446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:39.892606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e6a34-477a-4152-a9bd-8b36dd0e9446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:39.894096 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e6a34-477a-4152-a9bd-8b36dd0e9446 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:39.909980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 303/602] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:39.915835 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e6a34-477a-4152-a9bd-8b36dd0e9446 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:39.935552 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47514ade-f37a-43e8-b4b9-c890193a174e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:39.936776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47514ade-f37a-43e8-b4b9-c890193a174e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:39.937265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47514ade-f37a-43e8-b4b9-c890193a174e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:39.937720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47514ade-f37a-43e8-b4b9-c890193a174e None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:39.939254 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47514ade-f37a-43e8-b4b9-c890193a174e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:39.939756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47514ade-f37a-43e8-b4b9-c890193a174e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:39.941751 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47514ade-f37a-43e8-b4b9-c890193a174e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:39.959858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 300/603] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:39.966870 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47514ade-f37a-43e8-b4b9-c890193a174e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:40.052869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e6a34-477a-4152-a9bd-8b36dd0e9446 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:40.056454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:40.094924 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47514ade-f37a-43e8-b4b9-c890193a174e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:40.101273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3eef4-06f5-4ae8-87b3-2a9a25e1de0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:40.102158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3eef4-06f5-4ae8-87b3-2a9a25e1de0e None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:40.102662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3eef4-06f5-4ae8-87b3-2a9a25e1de0e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:40.103117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3eef4-06f5-4ae8-87b3-2a9a25e1de0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:40.110332 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3eef4-06f5-4ae8-87b3-2a9a25e1de0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:40.112050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3eef4-06f5-4ae8-87b3-2a9a25e1de0e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-04c3eef4-06f5-4ae8-87b3-2a9a25e1de0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:40.123662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:40.124544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:40.125859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:40.125859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:40.125859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:40.125859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:40.125859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:40.125859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:40.125859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:40.125859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:40.128244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 304/604] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:40.129890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed415c-bc66-4986-b8ab-d0d02a75be36 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:40.130433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed415c-bc66-4986-b8ab-d0d02a75be36 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:40.130972 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed415c-bc66-4986-b8ab-d0d02a75be36 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:40.132938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 305/605] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:40.133043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368767f-8f6e-4f75-a587-e2f89016a77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:40.133911 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed415c-bc66-4986-b8ab-d0d02a75be36 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:40.137987 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368767f-8f6e-4f75-a587-e2f89016a77f None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:40.138353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368767f-8f6e-4f75-a587-e2f89016a77f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:40.139705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368767f-8f6e-4f75-a587-e2f89016a77f None admin] PATH_INFO: `/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/95b665b9e27144b5bd684ece7ff465e2/roles/2b68b618a1b846f69bbe9aa08852a66f` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:40.142487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:40.159921 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368767f-8f6e-4f75-a587-e2f89016a77f None admin] RBAC: Authorizing `identity:create_grant(project_id=253ff30a56444df28d44c4b7eaadb276, user_id=95b665b9e27144b5bd684ece7ff465e2, role_id=2b68b618a1b846f69bbe9aa08852a66f)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:40.162324 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368767f-8f6e-4f75-a587-e2f89016a77f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:40.182700 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 301/606] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:18:39 2022] PUT /identity/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/95b665b9e27144b5bd684ece7ff465e2/roles/2b68b618a1b846f69bbe9aa08852a66f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:18:40.186636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8625aa-68cf-4a18-bc4d-2d60ed2985b0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:40.186636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8625aa-68cf-4a18-bc4d-2d60ed2985b0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:40.186636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8625aa-68cf-4a18-bc4d-2d60ed2985b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:40.207165 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-2b8625aa-68cf-4a18-bc4d-2d60ed2985b0 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:40.209283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2b8625aa-68cf-4a18-bc4d-2d60ed2985b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:40.292717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 302/607] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:40.352348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed415c-bc66-4986-b8ab-d0d02a75be36 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:40.402803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2af8c-5bcd-4c99-a472-e098c66f6881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:40.403432 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2af8c-5bcd-4c99-a472-e098c66f6881 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:40.403711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2af8c-5bcd-4c99-a472-e098c66f6881 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:40.403981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2af8c-5bcd-4c99-a472-e098c66f6881 None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:40.404688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2af8c-5bcd-4c99-a472-e098c66f6881 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:40.404990 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2af8c-5bcd-4c99-a472-e098c66f6881 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:40.406433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2af8c-5bcd-4c99-a472-e098c66f6881 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:40.420703 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 306/608] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:40.428117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8625aa-68cf-4a18-bc4d-2d60ed2985b0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:40.535911 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8625aa-68cf-4a18-bc4d-2d60ed2985b0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:40.550528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:40.571414 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d67e1-0a27-43d8-8b14-253222d8d047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:40.572567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d67e1-0a27-43d8-8b14-253222d8d047 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:40.573116 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d67e1-0a27-43d8-8b14-253222d8d047 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:40.573956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d67e1-0a27-43d8-8b14-253222d8d047 None admin] PATH_INFO: `/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/aa65b6854dc54e289cc3e8e0b3746dd4/roles/2b68b618a1b846f69bbe9aa08852a66f` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:40.592210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d67e1-0a27-43d8-8b14-253222d8d047 None admin] RBAC: Authorizing `identity:create_grant(project_id=253ff30a56444df28d44c4b7eaadb276, user_id=aa65b6854dc54e289cc3e8e0b3746dd4, role_id=2b68b618a1b846f69bbe9aa08852a66f)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:40.593788 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d67e1-0a27-43d8-8b14-253222d8d047 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:40.602184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2af8c-5bcd-4c99-a472-e098c66f6881 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:40.616865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 303/609] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:18:40 2022] PUT /identity/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/aa65b6854dc54e289cc3e8e0b3746dd4/roles/2b68b618a1b846f69bbe9aa08852a66f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:18:40.634369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:40.709627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48543e89-64a5-40f8-be04-988a3af4e538 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:40.709960 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48543e89-64a5-40f8-be04-988a3af4e538 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:40.710304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48543e89-64a5-40f8-be04-988a3af4e538 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:40.711636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 304/610] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:18:40.717706 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af251b3-52df-4a21-9cf0-1afd5d5df83a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:40.718051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af251b3-52df-4a21-9cf0-1afd5d5df83a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:40.718260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af251b3-52df-4a21-9cf0-1afd5d5df83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:40.724360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2af8c-5bcd-4c99-a472-e098c66f6881 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:40.754763 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-6af251b3-52df-4a21-9cf0-1afd5d5df83a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:40.756078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6af251b3-52df-4a21-9cf0-1afd5d5df83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:40.776455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac23ec-fa2e-46f7-b632-5e9de3bb87d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:40.777476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac23ec-fa2e-46f7-b632-5e9de3bb87d2 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:40.777944 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac23ec-fa2e-46f7-b632-5e9de3bb87d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:40.778449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac23ec-fa2e-46f7-b632-5e9de3bb87d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:40.806762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac23ec-fa2e-46f7-b632-5e9de3bb87d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-0dac23ec-fa2e-46f7-b632-5e9de3bb87d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:40.812052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:40.813016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:40.813830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:40.813830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:40.813830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:40.813830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:40.813830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:40.813830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:40.813830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:40.813830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:40.813830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:40.813830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:40.813830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:40.813830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:40.823025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 307/611] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:40.843924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac23ec-fa2e-46f7-b632-5e9de3bb87d2 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:40.892556 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 305/612] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 16:18:40.962391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:40.984219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac23ec-fa2e-46f7-b632-5e9de3bb87d2 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:41.034896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80d200-3bb0-4bf0-953c-a1bcba9c4a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:41.035503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80d200-3bb0-4bf0-953c-a1bcba9c4a78 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:41.035779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80d200-3bb0-4bf0-953c-a1bcba9c4a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:41.036046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80d200-3bb0-4bf0-953c-a1bcba9c4a78 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:41.036708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80d200-3bb0-4bf0-953c-a1bcba9c4a78 None admin] RBAC: Adding query filter params (name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:41.037009 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80d200-3bb0-4bf0-953c-a1bcba9c4a78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:41.038530 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80d200-3bb0-4bf0-953c-a1bcba9c4a78 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:41.056983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 308/613] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:41.065297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af251b3-52df-4a21-9cf0-1afd5d5df83a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:41.159248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80d200-3bb0-4bf0-953c-a1bcba9c4a78 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:41.264291 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:41.276145 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af251b3-52df-4a21-9cf0-1afd5d5df83a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:41.351676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d3c9b-d4f1-414d-8805-65f3d1b2ecd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:41.353943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d3c9b-d4f1-414d-8805-65f3d1b2ecd4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:41.355606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d3c9b-d4f1-414d-8805-65f3d1b2ecd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:41.356300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d3c9b-d4f1-414d-8805-65f3d1b2ecd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:41.358333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80d200-3bb0-4bf0-953c-a1bcba9c4a78 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:41.379550 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d3c9b-d4f1-414d-8805-65f3d1b2ecd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:41.382955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d3c9b-d4f1-414d-8805-65f3d1b2ecd4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-7f6d3c9b-d4f1-414d-8805-65f3d1b2ecd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:41.413100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:41.414153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:41.415122 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:41.415122 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:41.415122 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:41.420853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 306/614] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:41.429394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d3c9b-d4f1-414d-8805-65f3d1b2ecd4 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:41.437851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742536a-38f2-43d9-a338-265fe8227881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:41.438526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742536a-38f2-43d9-a338-265fe8227881 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:41.438808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742536a-38f2-43d9-a338-265fe8227881 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:41.439079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742536a-38f2-43d9-a338-265fe8227881 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:41.458916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742536a-38f2-43d9-a338-265fe8227881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-d742536a-38f2-43d9-a338-265fe8227881 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:41.466104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:41.467254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:41.468271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:41.468271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:41.468271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:41.468271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:41.468271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:41.468271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:41.468271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:41.468271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:41.468271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:41.468271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:41.468271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:41.468271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:41.474475 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 309/615] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:41.475644 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742536a-38f2-43d9-a338-265fe8227881 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:41.495323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:41.616597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d3c9b-d4f1-414d-8805-65f3d1b2ecd4 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:41.656736 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742536a-38f2-43d9-a338-265fe8227881 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:41.680902 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00846218-d0b8-487f-ba9c-a74ebcc63d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:41.681516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00846218-d0b8-487f-ba9c-a74ebcc63d1a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:41.681795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00846218-d0b8-487f-ba9c-a74ebcc63d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:41.682062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00846218-d0b8-487f-ba9c-a74ebcc63d1a None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:41.682734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00846218-d0b8-487f-ba9c-a74ebcc63d1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:41.683033 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00846218-d0b8-487f-ba9c-a74ebcc63d1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:41.684417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00846218-d0b8-487f-ba9c-a74ebcc63d1a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:41.701829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 307/616] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:41.703204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00846218-d0b8-487f-ba9c-a74ebcc63d1a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:41.712490 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:41.732115 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6408bd-d1e3-41a4-bd55-e5af18fb3848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:41.732741 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6408bd-d1e3-41a4-bd55-e5af18fb3848 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:41.733035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6408bd-d1e3-41a4-bd55-e5af18fb3848 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:41.733280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6408bd-d1e3-41a4-bd55-e5af18fb3848 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:41.733946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6408bd-d1e3-41a4-bd55-e5af18fb3848 None admin] RBAC: Adding query filter params (name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:41.734245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6408bd-d1e3-41a4-bd55-e5af18fb3848 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:41.735757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6408bd-d1e3-41a4-bd55-e5af18fb3848 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:41.759828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 310/617] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:41.762691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6408bd-d1e3-41a4-bd55-e5af18fb3848 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:41.822719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00846218-d0b8-487f-ba9c-a74ebcc63d1a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:41.874343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc7981-5911-43b2-94b8-2282caf433a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:41.875271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc7981-5911-43b2-94b8-2282caf433a8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:41.875701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc7981-5911-43b2-94b8-2282caf433a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:41.876117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc7981-5911-43b2-94b8-2282caf433a8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:41.891217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc7981-5911-43b2-94b8-2282caf433a8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:41.893215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc7981-5911-43b2-94b8-2282caf433a8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-79bc7981-5911-43b2-94b8-2282caf433a8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:41.918772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:41.919980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:41.921160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:41.921160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:41.921160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:41.921160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:41.921160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:41.921160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:41.921160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:41.921160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:41.921160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 16:18:41.921160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:41.928661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 308/618] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:18:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:41.932270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:41.936575 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc7981-5911-43b2-94b8-2282caf433a8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:41.967950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6408bd-d1e3-41a4-bd55-e5af18fb3848 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:42.025613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a164c7-6b8c-455c-9774-abc4ad1482d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:42.026254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a164c7-6b8c-455c-9774-abc4ad1482d0 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:42.026796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a164c7-6b8c-455c-9774-abc4ad1482d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:42.027073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a164c7-6b8c-455c-9774-abc4ad1482d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:42.049816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a164c7-6b8c-455c-9774-abc4ad1482d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:42.051306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a164c7-6b8c-455c-9774-abc4ad1482d0 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:42.064986 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-29a164c7-6b8c-455c-9774-abc4ad1482d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:42.087101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:42.088204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:42.089060 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:42.089060 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:42.089060 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:42.089698 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 311/619] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:42.095484 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a164c7-6b8c-455c-9774-abc4ad1482d0 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:42.143241 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc7981-5911-43b2-94b8-2282caf433a8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:42.206466 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6544bd-c688-4bd4-b34e-f3ac09b5b881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:42.207140 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6544bd-c688-4bd4-b34e-f3ac09b5b881 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:42.207416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6544bd-c688-4bd4-b34e-f3ac09b5b881 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:42.207682 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6544bd-c688-4bd4-b34e-f3ac09b5b881 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:42.208440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6544bd-c688-4bd4-b34e-f3ac09b5b881 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:42.208745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6544bd-c688-4bd4-b34e-f3ac09b5b881 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:42.210320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6544bd-c688-4bd4-b34e-f3ac09b5b881 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:42.250148 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 309/620] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Sep 9 16:18:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:42.259601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6544bd-c688-4bd4-b34e-f3ac09b5b881 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:42.299005 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a164c7-6b8c-455c-9774-abc4ad1482d0 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:42.323081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:42.344223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de12554-5a80-4010-a9b6-9e5f56acc383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:42.344847 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de12554-5a80-4010-a9b6-9e5f56acc383 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:42.345122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de12554-5a80-4010-a9b6-9e5f56acc383 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:42.345386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de12554-5a80-4010-a9b6-9e5f56acc383 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:42.346028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de12554-5a80-4010-a9b6-9e5f56acc383 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:42.346327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de12554-5a80-4010-a9b6-9e5f56acc383 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:42.347847 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de12554-5a80-4010-a9b6-9e5f56acc383 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:42.361103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 312/621] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:42.362300 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24199431-c507-4ba3-b790-8c198ea41012 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:42.362631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24199431-c507-4ba3-b790-8c198ea41012 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:42.362997 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24199431-c507-4ba3-b790-8c198ea41012 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:42.368404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 313/622] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:42.369238 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24199431-c507-4ba3-b790-8c198ea41012 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:42.414129 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6544bd-c688-4bd4-b34e-f3ac09b5b881 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:42.445282 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:42.460193 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65a107-8985-4c23-9e38-9772145a0ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:42.461551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65a107-8985-4c23-9e38-9772145a0ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:42.461975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65a107-8985-4c23-9e38-9772145a0ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:42.462398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65a107-8985-4c23-9e38-9772145a0ce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:42.476362 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65a107-8985-4c23-9e38-9772145a0ce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:42.478940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65a107-8985-4c23-9e38-9772145a0ce5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:42.487391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24199431-c507-4ba3-b790-8c198ea41012 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-cd65a107-8985-4c23-9e38-9772145a0ce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:42.498708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:42.499856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:42.500951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:42.500951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:42.500951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:42.503102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 310/623] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:42.505098 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25fb1c-f5b1-4a68-815a-81b7af264d21 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:42.505615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25fb1c-f5b1-4a68-815a-81b7af264d21 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:42.506095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25fb1c-f5b1-4a68-815a-81b7af264d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:42.524890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8c12c-f7bf-4acc-b4b7-c61197cec72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:42.525704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8c12c-f7bf-4acc-b4b7-c61197cec72c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:42.526081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8c12c-f7bf-4acc-b4b7-c61197cec72c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:42.526472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8c12c-f7bf-4acc-b4b7-c61197cec72c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:42.530035 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7b25fb1c-f5b1-4a68-815a-81b7af264d21 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:42.531424 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7b25fb1c-f5b1-4a68-815a-81b7af264d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:42.535459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8c12c-f7bf-4acc-b4b7-c61197cec72c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:42.536336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8c12c-f7bf-4acc-b4b7-c61197cec72c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-0db8c12c-f7bf-4acc-b4b7-c61197cec72c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:42.550086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:42.551358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:42.552522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:42.552522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:42.552522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:42.552522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:42.552522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:42.552522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:42.552522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:42.552522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:42.552522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:18:42.552522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:42.554372 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 314/624] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Sep 9 16:18:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:42.555497 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8c12c-f7bf-4acc-b4b7-c61197cec72c None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:42.601514 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:42.634653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 311/625] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:42.635851 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25fb1c-f5b1-4a68-815a-81b7af264d21 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:42.680900 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8c12c-f7bf-4acc-b4b7-c61197cec72c None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:42.716789 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b2828-05d7-490d-b1b8-fb49149c19cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:42.717870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b2828-05d7-490d-b1b8-fb49149c19cc None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:42.717870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b2828-05d7-490d-b1b8-fb49149c19cc None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:42.718094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b2828-05d7-490d-b1b8-fb49149c19cc None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:42.718675 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b2828-05d7-490d-b1b8-fb49149c19cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:42.718974 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b2828-05d7-490d-b1b8-fb49149c19cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:42.720384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b2828-05d7-490d-b1b8-fb49149c19cc None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:42.738993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 315/626] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:42.740286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57156121-659a-4847-a952-a071cbc86ef5 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:42.740697 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57156121-659a-4847-a952-a071cbc86ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:42.740967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57156121-659a-4847-a952-a071cbc86ef5 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:42.746640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 316/627] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:42.747461 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57156121-659a-4847-a952-a071cbc86ef5 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:42.757144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25fb1c-f5b1-4a68-815a-81b7af264d21 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:42.767304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:42.802270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f9bde-25bc-436d-afa1-76b8c2c830b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:42.803225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f9bde-25bc-436d-afa1-76b8c2c830b3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:42.803514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f9bde-25bc-436d-afa1-76b8c2c830b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:42.803782 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f9bde-25bc-436d-afa1-76b8c2c830b3 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:42.804912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f9bde-25bc-436d-afa1-76b8c2c830b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:42.805217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f9bde-25bc-436d-afa1-76b8c2c830b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:42.806712 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f9bde-25bc-436d-afa1-76b8c2c830b3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:42.833486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 312/628] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Sep 9 16:18:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:42.835754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7249f1c-4eb0-4b19-b7ce-7eacf8bb73ad None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:42.836054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7249f1c-4eb0-4b19-b7ce-7eacf8bb73ad None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:42.836326 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7249f1c-4eb0-4b19-b7ce-7eacf8bb73ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:42.858826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:42.859195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f7249f1c-4eb0-4b19-b7ce-7eacf8bb73ad None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:42.860353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f7249f1c-4eb0-4b19-b7ce-7eacf8bb73ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:42.866205 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57156121-659a-4847-a952-a071cbc86ef5 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:42.900473 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6b118-e06f-4cae-ba19-968bb440d75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:42.901162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6b118-e06f-4cae-ba19-968bb440d75a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:42.901444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6b118-e06f-4cae-ba19-968bb440d75a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:42.901704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6b118-e06f-4cae-ba19-968bb440d75a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:42.908713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6b118-e06f-4cae-ba19-968bb440d75a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:42.910352 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6b118-e06f-4cae-ba19-968bb440d75a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-3ad6b118-e06f-4cae-ba19-968bb440d75a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:42.921845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:42.922973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:42.923850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:42.923850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:42.923850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:42.923850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:42.923850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:42.923850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:42.923850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:42.923850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:42.927791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 317/629] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:42.928691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6b118-e06f-4cae-ba19-968bb440d75a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:42.950761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 313/630] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:42.950761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7249f1c-4eb0-4b19-b7ce-7eacf8bb73ad None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:43.041209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:43.046083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6b118-e06f-4cae-ba19-968bb440d75a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:43.048210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7249f1c-4eb0-4b19-b7ce-7eacf8bb73ad None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:43.091281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cf7f5-4e58-439a-b287-df8c32cd11af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:43.092608 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cf7f5-4e58-439a-b287-df8c32cd11af None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:43.092890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cf7f5-4e58-439a-b287-df8c32cd11af None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:43.093157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cf7f5-4e58-439a-b287-df8c32cd11af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:43.101355 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefabea3-0cf8-460f-9e2e-d68083b1739a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:43.102044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefabea3-0cf8-460f-9e2e-d68083b1739a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:43.102376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefabea3-0cf8-460f-9e2e-d68083b1739a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:43.102673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefabea3-0cf8-460f-9e2e-d68083b1739a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:43.113218 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefabea3-0cf8-460f-9e2e-d68083b1739a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-eefabea3-0cf8-460f-9e2e-d68083b1739a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:43.117982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:43.118925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:43.119748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:43.119748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:43.119748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:43.119748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:43.119748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:43.119748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:43.119748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:43.119748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:43.119748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:43.119748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:43.119748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:43.119748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:43.119748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 314/631] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:43.120428 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefabea3-0cf8-460f-9e2e-d68083b1739a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:43.129433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:43.177320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cf7f5-4e58-439a-b287-df8c32cd11af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:43.179684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cf7f5-4e58-439a-b287-df8c32cd11af None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:43.228713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefabea3-0cf8-460f-9e2e-d68083b1739a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:43.260533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfd11e-ceb5-4859-a2f5-d44452b79fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:43.261140 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfd11e-ceb5-4859-a2f5-d44452b79fab None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:43.261409 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfd11e-ceb5-4859-a2f5-d44452b79fab None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:43.261670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfd11e-ceb5-4859-a2f5-d44452b79fab None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:43.262349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfd11e-ceb5-4859-a2f5-d44452b79fab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:43.262707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfd11e-ceb5-4859-a2f5-d44452b79fab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:43.264120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfd11e-ceb5-4859-a2f5-d44452b79fab None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:43.275176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 315/632] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:43.275951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfd11e-ceb5-4859-a2f5-d44452b79fab None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:43.281980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 318/633] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:43.281980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11314c3b-b933-4e12-98f1-f50c0f134c1f None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:43.281980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11314c3b-b933-4e12-98f1-f50c0f134c1f None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:43.281980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11314c3b-b933-4e12-98f1-f50c0f134c1f None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:43.281980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 319/634] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:43.281980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11314c3b-b933-4e12-98f1-f50c0f134c1f None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:43.296727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:43.341739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:43.379785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfd11e-ceb5-4859-a2f5-d44452b79fab None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:43.382976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11314c3b-b933-4e12-98f1-f50c0f134c1f None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:43.412449 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742398f5-8471-4c3f-8eaf-6861b4afc7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:43.413056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742398f5-8471-4c3f-8eaf-6861b4afc7fa None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:43.413329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742398f5-8471-4c3f-8eaf-6861b4afc7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:43.413590 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742398f5-8471-4c3f-8eaf-6861b4afc7fa None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:43.414248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742398f5-8471-4c3f-8eaf-6861b4afc7fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:43.414842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742398f5-8471-4c3f-8eaf-6861b4afc7fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:43.415144 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd541be-f003-4806-b75c-3f64ce78b2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:43.415798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd541be-f003-4806-b75c-3f64ce78b2fd None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:43.415949 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742398f5-8471-4c3f-8eaf-6861b4afc7fa None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:43.416073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd541be-f003-4806-b75c-3f64ce78b2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:43.416340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd541be-f003-4806-b75c-3f64ce78b2fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:43.423908 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 316/635] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:43.424824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742398f5-8471-4c3f-8eaf-6861b4afc7fa None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:43.432004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd541be-f003-4806-b75c-3f64ce78b2fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-dcd541be-f003-4806-b75c-3f64ce78b2fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:43.437938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:43.439108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:43.440126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:43.440126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:43.440126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:43.440126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:43.440126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:43.440126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:43.440126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:43.440126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:43.440126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:43.440126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:43.440126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:43.440126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:43.442665 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 320/636] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:43.443562 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd541be-f003-4806-b75c-3f64ce78b2fd None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:43.501450 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:43.524194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742398f5-8471-4c3f-8eaf-6861b4afc7fa None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:43.540163 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd541be-f003-4806-b75c-3f64ce78b2fd None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:43.558080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27059eeb-ca71-4e98-8655-2c61b9c1312c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:43.558763 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27059eeb-ca71-4e98-8655-2c61b9c1312c None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:43.559039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27059eeb-ca71-4e98-8655-2c61b9c1312c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:43.559308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27059eeb-ca71-4e98-8655-2c61b9c1312c None admin] PATH_INFO: `/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/e926034945b44087b378b3468954a18a/roles/2b68b618a1b846f69bbe9aa08852a66f` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:43.560011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:43.573909 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27059eeb-ca71-4e98-8655-2c61b9c1312c None admin] RBAC: Authorizing `identity:create_grant(project_id=253ff30a56444df28d44c4b7eaadb276, user_id=e926034945b44087b378b3468954a18a, role_id=2b68b618a1b846f69bbe9aa08852a66f)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:43.574587 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065558a1-47a5-4741-a277-a6ac4e235890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:43.575216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065558a1-47a5-4741-a277-a6ac4e235890 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:43.575407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27059eeb-ca71-4e98-8655-2c61b9c1312c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:43.575524 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065558a1-47a5-4741-a277-a6ac4e235890 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:43.575762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065558a1-47a5-4741-a277-a6ac4e235890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:43.585805 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065558a1-47a5-4741-a277-a6ac4e235890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:43.587270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065558a1-47a5-4741-a277-a6ac4e235890 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:43.588011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 317/637] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:18:43 2022] PUT /identity/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/e926034945b44087b378b3468954a18a/roles/2b68b618a1b846f69bbe9aa08852a66f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:18:43.589233 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a089ae5-fc05-4c3f-a02d-8c6a7a681e17 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:43.589513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a089ae5-fc05-4c3f-a02d-8c6a7a681e17 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:43.589794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a089ae5-fc05-4c3f-a02d-8c6a7a681e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-065558a1-47a5-4741-a277-a6ac4e235890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:43.602771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:43.603998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:43.604859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:43.604859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:43.604859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:43.604859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 321/638] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:43.605086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065558a1-47a5-4741-a277-a6ac4e235890 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:43.620601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-9a089ae5-fc05-4c3f-a02d-8c6a7a681e17 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:43.622221 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9a089ae5-fc05-4c3f-a02d-8c6a7a681e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:43.695955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 318/639] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:43.696938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a089ae5-fc05-4c3f-a02d-8c6a7a681e17 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:43.738758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065558a1-47a5-4741-a277-a6ac4e235890 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:43.760880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:43.790351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69afb2b-a77a-47a1-b3f5-c523b6e8d068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:43.791046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69afb2b-a77a-47a1-b3f5-c523b6e8d068 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:43.791323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69afb2b-a77a-47a1-b3f5-c523b6e8d068 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:43.791594 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69afb2b-a77a-47a1-b3f5-c523b6e8d068 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:43.803196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69afb2b-a77a-47a1-b3f5-c523b6e8d068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:43.804384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69afb2b-a77a-47a1-b3f5-c523b6e8d068 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:43.816566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a089ae5-fc05-4c3f-a02d-8c6a7a681e17 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-b69afb2b-a77a-47a1-b3f5-c523b6e8d068 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:43.821392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:43.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:43.823570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:43.823570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:43.823570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:43.823570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 322/640] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:43.823570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69afb2b-a77a-47a1-b3f5-c523b6e8d068 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:43.861100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576641fb-5c53-47e7-80b3-70f9c4987281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:43.861718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576641fb-5c53-47e7-80b3-70f9c4987281 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:43.861992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576641fb-5c53-47e7-80b3-70f9c4987281 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:43.862256 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576641fb-5c53-47e7-80b3-70f9c4987281 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:43.863702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576641fb-5c53-47e7-80b3-70f9c4987281 None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:43.863702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576641fb-5c53-47e7-80b3-70f9c4987281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:43.864734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576641fb-5c53-47e7-80b3-70f9c4987281 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:43.871685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:43.875123 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 319/641] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:43.876100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576641fb-5c53-47e7-80b3-70f9c4987281 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:43.970227 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69afb2b-a77a-47a1-b3f5-c523b6e8d068 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:44.005693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd704454-01a4-46b7-9da5-62acc003dd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:44.006628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd704454-01a4-46b7-9da5-62acc003dd28 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:44.007000 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd704454-01a4-46b7-9da5-62acc003dd28 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:44.007363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd704454-01a4-46b7-9da5-62acc003dd28 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:44.008145 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd704454-01a4-46b7-9da5-62acc003dd28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:44.008540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd704454-01a4-46b7-9da5-62acc003dd28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:44.010008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd704454-01a4-46b7-9da5-62acc003dd28 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:44.026627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 323/642] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:44.026627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd704454-01a4-46b7-9da5-62acc003dd28 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:44.064272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576641fb-5c53-47e7-80b3-70f9c4987281 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:44.104114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:44.126334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:44.128911 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab448ad-04fe-4aec-b44c-1c80aed00857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:44.129895 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab448ad-04fe-4aec-b44c-1c80aed00857 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:44.130383 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab448ad-04fe-4aec-b44c-1c80aed00857 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:44.131168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab448ad-04fe-4aec-b44c-1c80aed00857 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:44.132747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab448ad-04fe-4aec-b44c-1c80aed00857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:44.135086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab448ad-04fe-4aec-b44c-1c80aed00857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:44.136504 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab448ad-04fe-4aec-b44c-1c80aed00857 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:44.148865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd704454-01a4-46b7-9da5-62acc003dd28 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:44.153908 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 320/643] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:44.153908 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab448ad-04fe-4aec-b44c-1c80aed00857 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:44.189104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af7d5a-7bb8-457c-a82c-81a5dea293ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:44.190032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af7d5a-7bb8-457c-a82c-81a5dea293ea None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:44.190714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af7d5a-7bb8-457c-a82c-81a5dea293ea None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:44.191709 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af7d5a-7bb8-457c-a82c-81a5dea293ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:44.205704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af7d5a-7bb8-457c-a82c-81a5dea293ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:44.209579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af7d5a-7bb8-457c-a82c-81a5dea293ea None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-73af7d5a-7bb8-457c-a82c-81a5dea293ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:44.231311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:44.232374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:44.233248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:44.233248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:44.233248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:44.233428 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 324/644] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:44.234211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af7d5a-7bb8-457c-a82c-81a5dea293ea None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:44.274701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab448ad-04fe-4aec-b44c-1c80aed00857 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:44.311361 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e267ea4-799c-4e67-afb5-b7f99fb949a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:44.312347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e267ea4-799c-4e67-afb5-b7f99fb949a8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:44.312821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e267ea4-799c-4e67-afb5-b7f99fb949a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:44.313304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e267ea4-799c-4e67-afb5-b7f99fb949a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:44.324642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e267ea4-799c-4e67-afb5-b7f99fb949a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-9e267ea4-799c-4e67-afb5-b7f99fb949a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:44.329382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:44.330289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:44.331495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:44.331495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:44.331495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:44.331495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:44.331495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:44.331495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:44.331495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:44.331495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:44.331495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:44.331495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:44.331495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:44.331495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:44.331495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 321/645] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:44.331495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e267ea4-799c-4e67-afb5-b7f99fb949a8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:44.363040 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af7d5a-7bb8-457c-a82c-81a5dea293ea None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:44.375773 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:44.410435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cdf32a-b330-4743-a2ce-8eb5983337ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:44.411099 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdf32a-b330-4743-a2ce-8eb5983337ae None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:44.411378 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdf32a-b330-4743-a2ce-8eb5983337ae None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:44.411650 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdf32a-b330-4743-a2ce-8eb5983337ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:44.414363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:44.419786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cdf32a-b330-4743-a2ce-8eb5983337ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:44.421685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cdf32a-b330-4743-a2ce-8eb5983337ae None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-d0cdf32a-b330-4743-a2ce-8eb5983337ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:44.434204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:44.435313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:44.436400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:44.436400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:44.436400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:44.436400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:44.436400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:44.436400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:44.436400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:44.436400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:44.437302 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 325/646] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:44.438092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cdf32a-b330-4743-a2ce-8eb5983337ae None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:44.455423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e267ea4-799c-4e67-afb5-b7f99fb949a8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:44.489277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09991764-58e6-4eb0-bb66-e3c6b73ff656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:44.489911 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09991764-58e6-4eb0-bb66-e3c6b73ff656 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:44.490188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09991764-58e6-4eb0-bb66-e3c6b73ff656 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:44.490485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09991764-58e6-4eb0-bb66-e3c6b73ff656 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:44.498072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09991764-58e6-4eb0-bb66-e3c6b73ff656 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:44.499668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09991764-58e6-4eb0-bb66-e3c6b73ff656 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-09991764-58e6-4eb0-bb66-e3c6b73ff656 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:44.511267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:44.512392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:44.513526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:44.513526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:44.513526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:44.513526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:44.513526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:44.513526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:44.513526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:44.513526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:44.513526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 16:18:44.513526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:44.519075 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 322/647] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:44.519075 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09991764-58e6-4eb0-bb66-e3c6b73ff656 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:44.546861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cdf32a-b330-4743-a2ce-8eb5983337ae None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:44.580857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a9f1c-237e-4518-82e8-a3059ccc6dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:44.581436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a9f1c-237e-4518-82e8-a3059ccc6dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:44.581713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a9f1c-237e-4518-82e8-a3059ccc6dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:44.581971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a9f1c-237e-4518-82e8-a3059ccc6dd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:44.582631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a9f1c-237e-4518-82e8-a3059ccc6dd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:44.582927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a9f1c-237e-4518-82e8-a3059ccc6dd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:44.584320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a9f1c-237e-4518-82e8-a3059ccc6dd4 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:44.599996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 326/648] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:44.599996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a9f1c-237e-4518-82e8-a3059ccc6dd4 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:44.627085 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:44.631921 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09991764-58e6-4eb0-bb66-e3c6b73ff656 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:44.653351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:44.672850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13628f85-9934-40f8-8dda-96929f90555b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:44.673693 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13628f85-9934-40f8-8dda-96929f90555b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:44.673969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13628f85-9934-40f8-8dda-96929f90555b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:44.674231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13628f85-9934-40f8-8dda-96929f90555b None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:44.674916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13628f85-9934-40f8-8dda-96929f90555b None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:44.675215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13628f85-9934-40f8-8dda-96929f90555b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:44.676628 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13628f85-9934-40f8-8dda-96929f90555b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:44.689330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 323/649] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:44.689330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13628f85-9934-40f8-8dda-96929f90555b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:44.703667 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a9f1c-237e-4518-82e8-a3059ccc6dd4 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:44.738902 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206e6fa-4662-431d-a022-86809d0277aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:44.739608 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206e6fa-4662-431d-a022-86809d0277aa None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:44.739960 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206e6fa-4662-431d-a022-86809d0277aa None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:44.740288 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206e6fa-4662-431d-a022-86809d0277aa None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:44.741095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206e6fa-4662-431d-a022-86809d0277aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:44.741460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206e6fa-4662-431d-a022-86809d0277aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:44.742992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206e6fa-4662-431d-a022-86809d0277aa None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:44.759554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 327/650] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:44.759554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206e6fa-4662-431d-a022-86809d0277aa None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:44.796531 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13628f85-9934-40f8-8dda-96929f90555b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:44.831677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f729e-b72c-4e60-b484-de1ad5948d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:44.832379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f729e-b72c-4e60-b484-de1ad5948d65 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:44.832704 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f729e-b72c-4e60-b484-de1ad5948d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:44.833033 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f729e-b72c-4e60-b484-de1ad5948d65 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:44.833820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f729e-b72c-4e60-b484-de1ad5948d65 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:44.834278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f729e-b72c-4e60-b484-de1ad5948d65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:44.835980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f729e-b72c-4e60-b484-de1ad5948d65 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:44.853719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 324/651] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Sep 9 16:18:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:44.855339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f729e-b72c-4e60-b484-de1ad5948d65 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:44.872483 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206e6fa-4662-431d-a022-86809d0277aa None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:44.880145 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:44.888533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:44.907280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c51cd-00b5-4058-96d3-b49cf0d06833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:44.908152 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c51cd-00b5-4058-96d3-b49cf0d06833 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:44.908512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c51cd-00b5-4058-96d3-b49cf0d06833 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:44.908834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c51cd-00b5-4058-96d3-b49cf0d06833 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:44.917176 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c51cd-00b5-4058-96d3-b49cf0d06833 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:44.918808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c51cd-00b5-4058-96d3-b49cf0d06833 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-aa1c51cd-00b5-4058-96d3-b49cf0d06833 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:44.930342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:44.931534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:44.932542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:44.932542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:44.932542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:44.932542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:44.932542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:44.932542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:44.932542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:44.932542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:44.932542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 16:18:44.932542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:44.932542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 328/652] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:44.933145 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c51cd-00b5-4058-96d3-b49cf0d06833 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:44.969289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f729e-b72c-4e60-b484-de1ad5948d65 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:45.004116 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b26d7-001e-4536-bec8-7ef33cd8e4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:45.005211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b26d7-001e-4536-bec8-7ef33cd8e4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:45.005487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b26d7-001e-4536-bec8-7ef33cd8e4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:45.005892 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b26d7-001e-4536-bec8-7ef33cd8e4e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:45.015660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b26d7-001e-4536-bec8-7ef33cd8e4e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:45.017089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b26d7-001e-4536-bec8-7ef33cd8e4e5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-ac9b26d7-001e-4536-bec8-7ef33cd8e4e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:45.031272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:45.032224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:45.033124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:45.033124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:45.033124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:45.033124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 325/653] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:45.033124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b26d7-001e-4536-bec8-7ef33cd8e4e5 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:45.044659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c51cd-00b5-4058-96d3-b49cf0d06833 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:45.096978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98243e0c-a442-4623-9c14-eefdb7684d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:45.097612 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98243e0c-a442-4623-9c14-eefdb7684d23 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:45.097891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98243e0c-a442-4623-9c14-eefdb7684d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:45.098360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98243e0c-a442-4623-9c14-eefdb7684d23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:45.099020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98243e0c-a442-4623-9c14-eefdb7684d23 None admin] RBAC: Adding query filter params (role.id=2b68b618a1b846f69bbe9aa08852a66f, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=152a41ad598b4b01b1881d928c53c848) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:45.099328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98243e0c-a442-4623-9c14-eefdb7684d23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:45.100826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98243e0c-a442-4623-9c14-eefdb7684d23 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:45.114539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 329/654] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:18:44 2022] GET /identity/v3/role_assignments?user.id=152a41ad598b4b01b1881d928c53c848&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=2b68b618a1b846f69bbe9aa08852a66f => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:45.114539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98243e0c-a442-4623-9c14-eefdb7684d23 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:45.117772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:45.156904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b26d7-001e-4536-bec8-7ef33cd8e4e5 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:45.168139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:45.201775 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab96082-1810-4acd-bc1f-5365258404ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:45.203051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab96082-1810-4acd-bc1f-5365258404ca None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:45.203327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab96082-1810-4acd-bc1f-5365258404ca None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:45.203887 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab96082-1810-4acd-bc1f-5365258404ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:45.248433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98243e0c-a442-4623-9c14-eefdb7684d23 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:45.293283 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80c0d1-5d4b-48ce-99ff-675e6c344256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:45.294010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80c0d1-5d4b-48ce-99ff-675e6c344256 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:45.294374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80c0d1-5d4b-48ce-99ff-675e6c344256 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:45.294753 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80c0d1-5d4b-48ce-99ff-675e6c344256 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:45.295568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80c0d1-5d4b-48ce-99ff-675e6c344256 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:45.295942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80c0d1-5d4b-48ce-99ff-675e6c344256 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:45.298410 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80c0d1-5d4b-48ce-99ff-675e6c344256 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:45.299949 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab96082-1810-4acd-bc1f-5365258404ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:45.302439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab96082-1810-4acd-bc1f-5365258404ca None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:45.319081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 330/655] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Sep 9 16:18:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:45.320061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80c0d1-5d4b-48ce-99ff-675e6c344256 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:45.392305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:45.431344 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 326/656] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:45.432453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab96082-1810-4acd-bc1f-5365258404ca None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:45.446678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:45.452277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80c0d1-5d4b-48ce-99ff-675e6c344256 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:45.499063 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb894da4-02a0-4df6-b81e-8b6557416c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:45.499734 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb894da4-02a0-4df6-b81e-8b6557416c12 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:45.500041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb894da4-02a0-4df6-b81e-8b6557416c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:45.500479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb894da4-02a0-4df6-b81e-8b6557416c12 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:45.500982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb894da4-02a0-4df6-b81e-8b6557416c12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:45.501289 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb894da4-02a0-4df6-b81e-8b6557416c12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:45.502691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb894da4-02a0-4df6-b81e-8b6557416c12 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:45.520485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 331/657] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:45.520485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb894da4-02a0-4df6-b81e-8b6557416c12 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:45.544824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab96082-1810-4acd-bc1f-5365258404ca None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:45.580638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07fea4-3c48-4ef9-8fc2-48819a18ea8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:45.581530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07fea4-3c48-4ef9-8fc2-48819a18ea8a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:45.581948 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07fea4-3c48-4ef9-8fc2-48819a18ea8a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:45.582407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07fea4-3c48-4ef9-8fc2-48819a18ea8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:45.628847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:45.639713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb894da4-02a0-4df6-b81e-8b6557416c12 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:45.661336 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07fea4-3c48-4ef9-8fc2-48819a18ea8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:45.662926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07fea4-3c48-4ef9-8fc2-48819a18ea8a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:45.675152 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa3bfa-8df7-49cf-b4f5-0063679074c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:45.676006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa3bfa-8df7-49cf-b4f5-0063679074c8 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:45.676424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa3bfa-8df7-49cf-b4f5-0063679074c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:45.676834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa3bfa-8df7-49cf-b4f5-0063679074c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:45.693544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa3bfa-8df7-49cf-b4f5-0063679074c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:45.695106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa3bfa-8df7-49cf-b4f5-0063679074c8 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-2efa3bfa-8df7-49cf-b4f5-0063679074c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:45.713011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:45.714090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:45.715019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:45.715019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:45.715019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:45.719025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 332/658] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:45.720352 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa3bfa-8df7-49cf-b4f5-0063679074c8 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:45.735957 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:45.781466 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 327/659] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:45.782845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07fea4-3c48-4ef9-8fc2-48819a18ea8a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:45.842433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa3bfa-8df7-49cf-b4f5-0063679074c8 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:45.857345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:45.877376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08db401-092a-4a11-b562-63d967fcb9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:45.878017 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08db401-092a-4a11-b562-63d967fcb9df None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:45.878290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08db401-092a-4a11-b562-63d967fcb9df None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:45.878585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08db401-092a-4a11-b562-63d967fcb9df None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:45.886163 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08db401-092a-4a11-b562-63d967fcb9df None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:45.887632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08db401-092a-4a11-b562-63d967fcb9df None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:45.890643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07fea4-3c48-4ef9-8fc2-48819a18ea8a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-e08db401-092a-4a11-b562-63d967fcb9df None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:45.898714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:45.899676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:45.900618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:45.900618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:45.900618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:45.900618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:45.900618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:45.900618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:45.900618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:45.900618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:45.900618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 16:18:45.900618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:45.900618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 333/660] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:45.900618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08db401-092a-4a11-b562-63d967fcb9df None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:45.930453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0039c-7331-4c06-b35a-58788004d59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:45.931105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0039c-7331-4c06-b35a-58788004d59f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:45.931445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0039c-7331-4c06-b35a-58788004d59f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:45.931775 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0039c-7331-4c06-b35a-58788004d59f None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:45.932478 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0039c-7331-4c06-b35a-58788004d59f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:45.932838 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0039c-7331-4c06-b35a-58788004d59f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:45.934288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0039c-7331-4c06-b35a-58788004d59f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:45.950873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 328/661] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:45.950873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e32dd-a767-4e7a-ad04-f9cfe955c402 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:45.950873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e32dd-a767-4e7a-ad04-f9cfe955c402 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:45.950873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e32dd-a767-4e7a-ad04-f9cfe955c402 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:45.950873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 329/662] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:45.950873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e32dd-a767-4e7a-ad04-f9cfe955c402 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:45.982614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:46.015531 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08db401-092a-4a11-b562-63d967fcb9df None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:46.049757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7883097-9e26-4c98-a710-b51b8e9d373f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:46.050396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7883097-9e26-4c98-a710-b51b8e9d373f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:46.050703 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7883097-9e26-4c98-a710-b51b8e9d373f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:46.050982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7883097-9e26-4c98-a710-b51b8e9d373f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:46.055263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e32dd-a767-4e7a-ad04-f9cfe955c402 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:46.061986 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7883097-9e26-4c98-a710-b51b8e9d373f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:46.063929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7883097-9e26-4c98-a710-b51b8e9d373f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-f7883097-9e26-4c98-a710-b51b8e9d373f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:46.079064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:46.080122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:46.080990 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:46.080990 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:46.080990 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:46.084809 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 334/663] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:46.085548 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7883097-9e26-4c98-a710-b51b8e9d373f None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:46.093906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:46.101204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6296bb6-e415-4a9a-ba01-a52b21a49c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:46.101824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6296bb6-e415-4a9a-ba01-a52b21a49c2f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:46.102054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6296bb6-e415-4a9a-ba01-a52b21a49c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:46.102339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6296bb6-e415-4a9a-ba01-a52b21a49c2f None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:46.103133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6296bb6-e415-4a9a-ba01-a52b21a49c2f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:46.103412 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6296bb6-e415-4a9a-ba01-a52b21a49c2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:46.104838 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6296bb6-e415-4a9a-ba01-a52b21a49c2f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:46.123102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 330/664] 10.222.0.23 () {56 vars in 1139 bytes} [Fri Sep 9 16:18:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:46.124174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdac3e-3597-4313-a3d7-1f2966fc5cb5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:46.124457 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdac3e-3597-4313-a3d7-1f2966fc5cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:46.124722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdac3e-3597-4313-a3d7-1f2966fc5cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:46.142884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-40cdac3e-3597-4313-a3d7-1f2966fc5cb5 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:46.144061 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-40cdac3e-3597-4313-a3d7-1f2966fc5cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:46.200493 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7883097-9e26-4c98-a710-b51b8e9d373f None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:46.218549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 331/665] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:46.218549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cdac3e-3597-4313-a3d7-1f2966fc5cb5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:46.238126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:46.254464 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f99914-266b-4397-ba8e-f3b01638d605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:46.255496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f99914-266b-4397-ba8e-f3b01638d605 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:46.256128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f99914-266b-4397-ba8e-f3b01638d605 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:46.256597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f99914-266b-4397-ba8e-f3b01638d605 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:46.264899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f99914-266b-4397-ba8e-f3b01638d605 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:46.267122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f99914-266b-4397-ba8e-f3b01638d605 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-25f99914-266b-4397-ba8e-f3b01638d605 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:46.281739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:46.282686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:46.283571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:46.283571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:46.283571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:46.283571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:46.283571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:46.283571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:46.283571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:46.283571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:46.289978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 335/666] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:46.289978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f99914-266b-4397-ba8e-f3b01638d605 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:46.352889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cdac3e-3597-4313-a3d7-1f2966fc5cb5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:46.374586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:46.393852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f99914-266b-4397-ba8e-f3b01638d605 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:46.399079 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500da9f-51f0-47f8-8c64-23f43317b2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:46.399875 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500da9f-51f0-47f8-8c64-23f43317b2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:46.400330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500da9f-51f0-47f8-8c64-23f43317b2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:46.400733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500da9f-51f0-47f8-8c64-23f43317b2c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:46.401555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500da9f-51f0-47f8-8c64-23f43317b2c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:46.402600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500da9f-51f0-47f8-8c64-23f43317b2c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:46.404555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500da9f-51f0-47f8-8c64-23f43317b2c0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:46.423607 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 332/667] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:46.425114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500da9f-51f0-47f8-8c64-23f43317b2c0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:46.441330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349c577-722f-4f48-86cc-edc124f9a058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:46.442337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349c577-722f-4f48-86cc-edc124f9a058 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:46.442921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349c577-722f-4f48-86cc-edc124f9a058 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:46.443191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349c577-722f-4f48-86cc-edc124f9a058 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:46.500089 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:46.519310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349c577-722f-4f48-86cc-edc124f9a058 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:46.520895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349c577-722f-4f48-86cc-edc124f9a058 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:46.548372 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500da9f-51f0-47f8-8c64-23f43317b2c0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:46.584452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f9c2e-549e-42a3-afbf-1018ac3f1463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:46.585001 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f9c2e-549e-42a3-afbf-1018ac3f1463 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:46.585271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f9c2e-549e-42a3-afbf-1018ac3f1463 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:46.585536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f9c2e-549e-42a3-afbf-1018ac3f1463 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:46.586214 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f9c2e-549e-42a3-afbf-1018ac3f1463 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:46.586543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f9c2e-549e-42a3-afbf-1018ac3f1463 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:46.587973 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f9c2e-549e-42a3-afbf-1018ac3f1463 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:46.604585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 333/668] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:46.604585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f9c2e-549e-42a3-afbf-1018ac3f1463 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:46.632058 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 336/669] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:46.637449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349c577-722f-4f48-86cc-edc124f9a058 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:46.645520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:46.708601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f9c2e-549e-42a3-afbf-1018ac3f1463 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:46.724315 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:46.745131 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349c577-722f-4f48-86cc-edc124f9a058 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:46.746247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07c6c5-7163-4938-b228-3c3f011b8783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:46.746926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07c6c5-7163-4938-b228-3c3f011b8783 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:46.747198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07c6c5-7163-4938-b228-3c3f011b8783 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:46.747461 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07c6c5-7163-4938-b228-3c3f011b8783 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:46.754440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07c6c5-7163-4938-b228-3c3f011b8783 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:46.756150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07c6c5-7163-4938-b228-3c3f011b8783 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-9e07c6c5-7163-4938-b228-3c3f011b8783 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:46.767586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:46.768647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:46.769722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:46.769722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:46.769722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:46.769722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:46.769722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:46.769722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:46.769722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:46.769722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:46.773653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 334/670] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:46.774475 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07c6c5-7163-4938-b228-3c3f011b8783 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:46.789564 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf8f3a-9019-400d-b1d8-0ff1ac0f1c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:46.790194 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf8f3a-9019-400d-b1d8-0ff1ac0f1c1b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:46.790466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf8f3a-9019-400d-b1d8-0ff1ac0f1c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:46.791093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf8f3a-9019-400d-b1d8-0ff1ac0f1c1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:46.801329 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf8f3a-9019-400d-b1d8-0ff1ac0f1c1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-3faf8f3a-9019-400d-b1d8-0ff1ac0f1c1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:46.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:46.807786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:46.809138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:46.809138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:46.809138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:46.809138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:46.809138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:46.809138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:46.809138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:46.809138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:46.809138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:46.809138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:46.809138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:46.809138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:46.809138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 337/671] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:46.813226 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf8f3a-9019-400d-b1d8-0ff1ac0f1c1b None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:46.880194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:46.885684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07c6c5-7163-4938-b228-3c3f011b8783 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:46.919754 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf8f3a-9019-400d-b1d8-0ff1ac0f1c1b None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:46.928026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b413c1-296b-441a-9b1e-60097478c12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:46.928832 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b413c1-296b-441a-9b1e-60097478c12f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:46.929112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b413c1-296b-441a-9b1e-60097478c12f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:46.929375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b413c1-296b-441a-9b1e-60097478c12f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:46.930091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b413c1-296b-441a-9b1e-60097478c12f None admin] RBAC: Adding query filter params (role.id=2b68b618a1b846f69bbe9aa08852a66f, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=62a2fb70c9ae4f4b81ac4de513b868cc) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:46.930376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b413c1-296b-441a-9b1e-60097478c12f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:46.931835 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b413c1-296b-441a-9b1e-60097478c12f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:46.950527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 335/672] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:18:46 2022] GET /identity/v3/role_assignments?user.id=62a2fb70c9ae4f4b81ac4de513b868cc&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=2b68b618a1b846f69bbe9aa08852a66f => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:46.950527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b413c1-296b-441a-9b1e-60097478c12f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:46.967323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300478c-b117-4202-bac6-b2b0b32d38fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:46.968195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300478c-b117-4202-bac6-b2b0b32d38fd None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:46.968552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300478c-b117-4202-bac6-b2b0b32d38fd None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:46.968895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300478c-b117-4202-bac6-b2b0b32d38fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:46.982360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300478c-b117-4202-bac6-b2b0b32d38fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:46.984037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300478c-b117-4202-bac6-b2b0b32d38fd None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:46.998902 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-b300478c-b117-4202-bac6-b2b0b32d38fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:47.006598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:47.007802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:47.008893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:47.008893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:47.008893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:47.010382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 338/673] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:47.011680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300478c-b117-4202-bac6-b2b0b32d38fd None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:47.073764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b413c1-296b-441a-9b1e-60097478c12f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:47.113811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a79d9-4d6b-4256-aee4-e3d62dad1a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:47.114897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a79d9-4d6b-4256-aee4-e3d62dad1a5e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:47.115351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a79d9-4d6b-4256-aee4-e3d62dad1a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:47.115969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a79d9-4d6b-4256-aee4-e3d62dad1a5e None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:47.116858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a79d9-4d6b-4256-aee4-e3d62dad1a5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:47.117348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a79d9-4d6b-4256-aee4-e3d62dad1a5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:47.119295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a79d9-4d6b-4256-aee4-e3d62dad1a5e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:47.138824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 336/674] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:47.140189 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a79d9-4d6b-4256-aee4-e3d62dad1a5e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:47.142833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300478c-b117-4202-bac6-b2b0b32d38fd None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:47.168971 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:47.187420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfaf8fa-4bf6-4dce-b6f3-c716126e3410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:47.188242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaf8fa-4bf6-4dce-b6f3-c716126e3410 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:47.188617 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaf8fa-4bf6-4dce-b6f3-c716126e3410 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:47.188976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaf8fa-4bf6-4dce-b6f3-c716126e3410 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:47.189846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfaf8fa-4bf6-4dce-b6f3-c716126e3410 None admin] RBAC: Adding query filter params (name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:47.190243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfaf8fa-4bf6-4dce-b6f3-c716126e3410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:47.192871 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfaf8fa-4bf6-4dce-b6f3-c716126e3410 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:47.204206 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 339/675] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:47.204980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfaf8fa-4bf6-4dce-b6f3-c716126e3410 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:47.264397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a79d9-4d6b-4256-aee4-e3d62dad1a5e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:47.271277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:47.305269 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a6dc7-c980-4954-8e61-12d017d404ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:47.306233 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a6dc7-c980-4954-8e61-12d017d404ad None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:47.307078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a6dc7-c980-4954-8e61-12d017d404ad None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:47.307078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a6dc7-c980-4954-8e61-12d017d404ad None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:47.307528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a6dc7-c980-4954-8e61-12d017d404ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:47.307835 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a6dc7-c980-4954-8e61-12d017d404ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:47.309261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a6dc7-c980-4954-8e61-12d017d404ad None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:47.322764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 337/676] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:47.322945 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfaf8fa-4bf6-4dce-b6f3-c716126e3410 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:47.326614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a6dc7-c980-4954-8e61-12d017d404ad None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:47.362236 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbbaec-9a99-4859-a540-fca262b9fc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:47.363107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbbaec-9a99-4859-a540-fca262b9fc6a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:47.363455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbbaec-9a99-4859-a540-fca262b9fc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:47.363771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbbaec-9a99-4859-a540-fca262b9fc6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:47.364589 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbbaec-9a99-4859-a540-fca262b9fc6a None admin] RBAC: Adding query filter params (role.id=2b68b618a1b846f69bbe9aa08852a66f, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=95b665b9e27144b5bd684ece7ff465e2) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:47.365265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbbaec-9a99-4859-a540-fca262b9fc6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:47.367663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbbaec-9a99-4859-a540-fca262b9fc6a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:47.374441 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 340/677] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:18:47 2022] GET /identity/v3/role_assignments?user.id=95b665b9e27144b5bd684ece7ff465e2&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=2b68b618a1b846f69bbe9aa08852a66f => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:47.375257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbbaec-9a99-4859-a540-fca262b9fc6a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:47.450876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:47.489182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a6dc7-c980-4954-8e61-12d017d404ad None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:47.508063 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbbaec-9a99-4859-a540-fca262b9fc6a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:47.532093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:47.549891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9f6a5-32c9-405b-9133-02d303363398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:47.551049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9f6a5-32c9-405b-9133-02d303363398 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:47.551478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9f6a5-32c9-405b-9133-02d303363398 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:47.551920 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a3fb9-0739-46c2-8152-8a586eaed5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:47.552408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9f6a5-32c9-405b-9133-02d303363398 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:47.552814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a3fb9-0739-46c2-8152-8a586eaed5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:47.553202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a3fb9-0739-46c2-8152-8a586eaed5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:47.553569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a3fb9-0739-46c2-8152-8a586eaed5b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:47.561121 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9f6a5-32c9-405b-9133-02d303363398 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:47.562901 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9f6a5-32c9-405b-9133-02d303363398 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:47.567841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a3fb9-0739-46c2-8152-8a586eaed5b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:47.569660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a3fb9-0739-46c2-8152-8a586eaed5b1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-8cd9f6a5-32c9-405b-9133-02d303363398 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:47.578362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:47.579569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:47.580684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:47.580684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:47.580684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:47.580684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:47.580684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:47.580684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:47.580684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:47.580684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:47.583615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 341/678] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:47.585351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0a747-bbfa-49a4-98fa-dd9e4c7c1f81 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:47.585781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0a747-bbfa-49a4-98fa-dd9e4c7c1f81 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:47.586216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0a747-bbfa-49a4-98fa-dd9e4c7c1f81 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:47.587908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 342/679] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:47.588765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0a747-bbfa-49a4-98fa-dd9e4c7c1f81 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-164a3fb9-0739-46c2-8152-8a586eaed5b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:47.592452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:47.593401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:47.594419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:47.594419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:47.594419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:47.594419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 338/680] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:47.598194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e67a2-3071-4876-9a44-e217d8e4cc8e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:47.598194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e67a2-3071-4876-9a44-e217d8e4cc8e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:47.598378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e67a2-3071-4876-9a44-e217d8e4cc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:47.621405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-955e67a2-3071-4876-9a44-e217d8e4cc8e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:47.622834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-955e67a2-3071-4876-9a44-e217d8e4cc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:47.706524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 339/681] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:47.706524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e67a2-3071-4876-9a44-e217d8e4cc8e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:47.743224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0a747-bbfa-49a4-98fa-dd9e4c7c1f81 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:47.785781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:47.831824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10471d2-8ade-4434-8850-defcc8344ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:47.832459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10471d2-8ade-4434-8850-defcc8344ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:47.832742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10471d2-8ade-4434-8850-defcc8344ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:47.833016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10471d2-8ade-4434-8850-defcc8344ac6 None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:47.833723 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10471d2-8ade-4434-8850-defcc8344ac6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:47.834066 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10471d2-8ade-4434-8850-defcc8344ac6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:47.834305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e67a2-3071-4876-9a44-e217d8e4cc8e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:47.835579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10471d2-8ade-4434-8850-defcc8344ac6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:47.865005 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 343/682] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:47.872995 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10471d2-8ade-4434-8850-defcc8344ac6 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:47.881533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9930d7-0818-4f71-949a-f41a207cea31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:47.882352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9930d7-0818-4f71-949a-f41a207cea31 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:47.882801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9930d7-0818-4f71-949a-f41a207cea31 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:47.883205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9930d7-0818-4f71-949a-f41a207cea31 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:47.884084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9930d7-0818-4f71-949a-f41a207cea31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:47.884561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9930d7-0818-4f71-949a-f41a207cea31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:47.886081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9930d7-0818-4f71-949a-f41a207cea31 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:47.907069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 340/683] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:47.912850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9930d7-0818-4f71-949a-f41a207cea31 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:47.960814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:48.028224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9930d7-0818-4f71-949a-f41a207cea31 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:48.054588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:48.068255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b6f74-19ce-4cde-acc7-92a690964609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:48.069313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b6f74-19ce-4cde-acc7-92a690964609 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:48.069729 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b6f74-19ce-4cde-acc7-92a690964609 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:48.070168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b6f74-19ce-4cde-acc7-92a690964609 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:48.079002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b6f74-19ce-4cde-acc7-92a690964609 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:48.080634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b6f74-19ce-4cde-acc7-92a690964609 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-9a0b6f74-19ce-4cde-acc7-92a690964609 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:48.093799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:48.094794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:48.096289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:48.096289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:48.096289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:48.096289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:48.096289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:48.096289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:48.096289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:48.096289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:48.096289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 16:18:48.096289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:48.096289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 341/684] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:18:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:48.096289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b6f74-19ce-4cde-acc7-92a690964609 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:48.098724 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10471d2-8ade-4434-8850-defcc8344ac6 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:48.137597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7c260-fef3-4f85-86c8-74396492319b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:48.138207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7c260-fef3-4f85-86c8-74396492319b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:48.138490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7c260-fef3-4f85-86c8-74396492319b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:48.138785 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7c260-fef3-4f85-86c8-74396492319b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:48.139608 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7c260-fef3-4f85-86c8-74396492319b None admin] RBAC: Adding query filter params (role.id=2b68b618a1b846f69bbe9aa08852a66f, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=aa65b6854dc54e289cc3e8e0b3746dd4) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:48.139917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7c260-fef3-4f85-86c8-74396492319b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:48.141441 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7c260-fef3-4f85-86c8-74396492319b None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:48.155100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 344/685] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:18:47 2022] GET /identity/v3/role_assignments?user.id=aa65b6854dc54e289cc3e8e0b3746dd4&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=2b68b618a1b846f69bbe9aa08852a66f => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:48.156206 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7c260-fef3-4f85-86c8-74396492319b None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:48.202279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b6f74-19ce-4cde-acc7-92a690964609 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:48.237797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec6b35-a00c-43b6-9300-fc6dfdf386f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:48.238439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec6b35-a00c-43b6-9300-fc6dfdf386f8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:48.238735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec6b35-a00c-43b6-9300-fc6dfdf386f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:48.239001 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec6b35-a00c-43b6-9300-fc6dfdf386f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:48.250363 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec6b35-a00c-43b6-9300-fc6dfdf386f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-f0ec6b35-a00c-43b6-9300-fc6dfdf386f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:48.255844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:48.258156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:48.260146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:48.260146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:48.260146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:48.260146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:48.260146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:48.260146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:48.260146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:48.260146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:48.260146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:48.260146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:48.260146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:18:48.260146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:48.260146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 342/686] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:18:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:48.266941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec6b35-a00c-43b6-9300-fc6dfdf386f8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:48.355108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7c260-fef3-4f85-86c8-74396492319b None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:48.358585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:48.359375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:48.398415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b4298-4e1c-40d1-b1a3-67881219a85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:48.399069 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4298-4e1c-40d1-b1a3-67881219a85e None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:48.399346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4298-4e1c-40d1-b1a3-67881219a85e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:48.399612 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4298-4e1c-40d1-b1a3-67881219a85e None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:48.400350 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b4298-4e1c-40d1-b1a3-67881219a85e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:48.400650 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b4298-4e1c-40d1-b1a3-67881219a85e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:48.402089 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b4298-4e1c-40d1-b1a3-67881219a85e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:48.425263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 345/687] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Sep 9 16:18:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:48.430870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b4298-4e1c-40d1-b1a3-67881219a85e None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:48.455213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec6b35-a00c-43b6-9300-fc6dfdf386f8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:48.508454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bd87e-c9af-471c-83fe-c7d193fa4ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:48.509073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bd87e-c9af-471c-83fe-c7d193fa4ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:48.509345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bd87e-c9af-471c-83fe-c7d193fa4ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:48.509614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bd87e-c9af-471c-83fe-c7d193fa4ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:48.510523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6bd87e-c9af-471c-83fe-c7d193fa4ff6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:48.511969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6bd87e-c9af-471c-83fe-c7d193fa4ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:48.513356 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6bd87e-c9af-471c-83fe-c7d193fa4ff6 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:48.522466 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 343/688] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:48.529234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bd87e-c9af-471c-83fe-c7d193fa4ff6 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:48.608761 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b4298-4e1c-40d1-b1a3-67881219a85e None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:48.629212 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bd87e-c9af-471c-83fe-c7d193fa4ff6 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:48.637285 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:48.664485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc37e2-6ed9-43e8-a4dd-bcdab621ddea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:48.665305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc37e2-6ed9-43e8-a4dd-bcdab621ddea None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:48.665672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc37e2-6ed9-43e8-a4dd-bcdab621ddea None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:48.666031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc37e2-6ed9-43e8-a4dd-bcdab621ddea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:48.676386 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc37e2-6ed9-43e8-a4dd-bcdab621ddea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:48.678073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc37e2-6ed9-43e8-a4dd-bcdab621ddea None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:48.692095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa928c-2b7a-430f-9e39-b453061d5323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:48.692726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa928c-2b7a-430f-9e39-b453061d5323 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:48.693002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa928c-2b7a-430f-9e39-b453061d5323 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:48.693269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa928c-2b7a-430f-9e39-b453061d5323 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-1fcc37e2-6ed9-43e8-a4dd-bcdab621ddea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:48.694327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:48.695328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:48.696209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:48.696209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:48.696209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:48.696209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 344/689] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:48.701852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc37e2-6ed9-43e8-a4dd-bcdab621ddea None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:48.784605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:48.808012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc37e2-6ed9-43e8-a4dd-bcdab621ddea None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:48.843812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb5c07-d2f0-413a-ac3c-bc24f6090e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:48.844526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb5c07-d2f0-413a-ac3c-bc24f6090e66 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:48.844862 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb5c07-d2f0-413a-ac3c-bc24f6090e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:48.845195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb5c07-d2f0-413a-ac3c-bc24f6090e66 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:48.845897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb5c07-d2f0-413a-ac3c-bc24f6090e66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:48.846258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb5c07-d2f0-413a-ac3c-bc24f6090e66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:48.848093 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb5c07-d2f0-413a-ac3c-bc24f6090e66 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:48.861354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 345/690] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:48.866948 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb5c07-d2f0-413a-ac3c-bc24f6090e66 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:48.889543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:48.905609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa928c-2b7a-430f-9e39-b453061d5323 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:48.907433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa928c-2b7a-430f-9e39-b453061d5323 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:48.972313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb5c07-d2f0-413a-ac3c-bc24f6090e66 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:49.007965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482f2dd-f613-4864-80df-5a6926357008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:49.008610 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482f2dd-f613-4864-80df-5a6926357008 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:49.008883 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482f2dd-f613-4864-80df-5a6926357008 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:49.009154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482f2dd-f613-4864-80df-5a6926357008 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:49.021324 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482f2dd-f613-4864-80df-5a6926357008 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:49.022820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482f2dd-f613-4864-80df-5a6926357008 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-1482f2dd-f613-4864-80df-5a6926357008 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:49.035329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:49.036331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:49.037319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:49.037319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:49.037319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:49.037319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:49.037319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:49.037319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:49.037319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:49.037319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:49.037319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:18:49.037319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:49.037319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 346/691] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Sep 9 16:18:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:49.042632 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482f2dd-f613-4864-80df-5a6926357008 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:49.142962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:49.166774 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482f2dd-f613-4864-80df-5a6926357008 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:49.179006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 346/692] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 749 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:49.185044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa928c-2b7a-430f-9e39-b453061d5323 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:49.239617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a7987-0336-46b8-b162-8688a0a2338f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:49.240270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a7987-0336-46b8-b162-8688a0a2338f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:49.240544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a7987-0336-46b8-b162-8688a0a2338f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:49.240810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a7987-0336-46b8-b162-8688a0a2338f None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:49.241546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a7987-0336-46b8-b162-8688a0a2338f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:49.241843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a7987-0336-46b8-b162-8688a0a2338f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:49.243304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a7987-0336-46b8-b162-8688a0a2338f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:49.270731 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 347/693] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Sep 9 16:18:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:49.276379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a7987-0336-46b8-b162-8688a0a2338f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:49.291715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:49.364498 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa928c-2b7a-430f-9e39-b453061d5323 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:49.445473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a7987-0336-46b8-b162-8688a0a2338f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:49.459845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b93b0-0e3b-495b-a130-f74ccd604de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:49.460501 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b93b0-0e3b-495b-a130-f74ccd604de9 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:49.460776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b93b0-0e3b-495b-a130-f74ccd604de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:49.461041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b93b0-0e3b-495b-a130-f74ccd604de9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:49.479799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61b72a-877c-48ef-aa74-562958897ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:49.480427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61b72a-877c-48ef-aa74-562958897ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:49.480698 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61b72a-877c-48ef-aa74-562958897ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:49.480962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61b72a-877c-48ef-aa74-562958897ec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:49.490905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:49.495692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b93b0-0e3b-495b-a130-f74ccd604de9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:49.497188 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b93b0-0e3b-495b-a130-f74ccd604de9 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-205b93b0-0e3b-495b-a130-f74ccd604de9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:49.535831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:49.536870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:49.537816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:49.537816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:49.537816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:49.539544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 347/694] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:49.547547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b93b0-0e3b-495b-a130-f74ccd604de9 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:49.559151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61b72a-877c-48ef-aa74-562958897ec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:49.560705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61b72a-877c-48ef-aa74-562958897ec7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:49.660674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 348/695] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 16:18:49.666161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61b72a-877c-48ef-aa74-562958897ec7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:49.722238 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b93b0-0e3b-495b-a130-f74ccd604de9 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:49.726888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:49.743555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:49.770416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b16d6e-2b2d-4859-8648-c315664783bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:49.771305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b16d6e-2b2d-4859-8648-c315664783bc None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:49.771642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b16d6e-2b2d-4859-8648-c315664783bc None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:49.772532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b16d6e-2b2d-4859-8648-c315664783bc None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:49.773191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b16d6e-2b2d-4859-8648-c315664783bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:49.773831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b16d6e-2b2d-4859-8648-c315664783bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:49.775354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b16d6e-2b2d-4859-8648-c315664783bc None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:49.788598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 348/696] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:49.790714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d710dc-cc43-4b7e-97c5-cebe1f7f78cc None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:49.790999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d710dc-cc43-4b7e-97c5-cebe1f7f78cc None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:49.791538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d710dc-cc43-4b7e-97c5-cebe1f7f78cc None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:49.793112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 349/697] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:49.799978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8d807-87fa-4ea4-9eb7-6e5d5b3ff24a None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:49.800266 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8d807-87fa-4ea4-9eb7-6e5d5b3ff24a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:49.800525 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8d807-87fa-4ea4-9eb7-6e5d5b3ff24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:49.813657 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61b72a-877c-48ef-aa74-562958897ec7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:49.822417 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-07a8d807-87fa-4ea4-9eb7-6e5d5b3ff24a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:49.824252 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-07a8d807-87fa-4ea4-9eb7-6e5d5b3ff24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:49.852618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac72521-8482-4907-b2bd-57504ea8ef6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:49.853246 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac72521-8482-4907-b2bd-57504ea8ef6d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:49.853521 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac72521-8482-4907-b2bd-57504ea8ef6d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:49.853788 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac72521-8482-4907-b2bd-57504ea8ef6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:49.864065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac72521-8482-4907-b2bd-57504ea8ef6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:49.865537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac72521-8482-4907-b2bd-57504ea8ef6d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-0ac72521-8482-4907-b2bd-57504ea8ef6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:49.879743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:49.880676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:49.881676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:49.881676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:49.881676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:49.881676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 349/698] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:49.881676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac72521-8482-4907-b2bd-57504ea8ef6d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:49.907566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 350/699] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:49.908757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65689c97-cdcc-4605-aaca-ca829f1eaf44 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:49.909045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65689c97-cdcc-4605-aaca-ca829f1eaf44 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:49.909313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65689c97-cdcc-4605-aaca-ca829f1eaf44 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:49.910720 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 351/700] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:49.911300 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65689c97-cdcc-4605-aaca-ca829f1eaf44 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:49.986389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:50.002007 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac72521-8482-4907-b2bd-57504ea8ef6d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:50.041120 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65689c97-cdcc-4605-aaca-ca829f1eaf44 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:50.049038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:50.049746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171d427-19df-4638-81b9-9496d00e44f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:50.050389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171d427-19df-4638-81b9-9496d00e44f1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:50.050696 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171d427-19df-4638-81b9-9496d00e44f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:50.051664 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171d427-19df-4638-81b9-9496d00e44f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:50.060179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171d427-19df-4638-81b9-9496d00e44f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:50.062096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171d427-19df-4638-81b9-9496d00e44f1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-b171d427-19df-4638-81b9-9496d00e44f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:50.079097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:50.080014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:50.080865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:50.080865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:50.080865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:50.080865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:50.080865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:50.080865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:50.080865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:50.080865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:50.085595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 350/701] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:50.087039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eca52-26a5-4d86-a9c7-6c3c814c8dc6 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:50.087398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eca52-26a5-4d86-a9c7-6c3c814c8dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:50.087745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eca52-26a5-4d86-a9c7-6c3c814c8dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:50.087952 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106dd67-60ac-443e-a126-01bc834bb181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:50.088574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106dd67-60ac-443e-a126-01bc834bb181 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:50.088854 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106dd67-60ac-443e-a126-01bc834bb181 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:50.089122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106dd67-60ac-443e-a126-01bc834bb181 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:50.089781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106dd67-60ac-443e-a126-01bc834bb181 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:50.090079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106dd67-60ac-443e-a126-01bc834bb181 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:50.091478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106dd67-60ac-443e-a126-01bc834bb181 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:50.104587 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 352/702] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:50.105531 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106dd67-60ac-443e-a126-01bc834bb181 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:50.111589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b25eca52-26a5-4d86-a9c7-6c3c814c8dc6 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:50.112922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b25eca52-26a5-4d86-a9c7-6c3c814c8dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:50.198583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 351/703] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:50.199624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25eca52-26a5-4d86-a9c7-6c3c814c8dc6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:50.229422 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106dd67-60ac-443e-a126-01bc834bb181 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:50.270833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916b33b-cf6f-4d48-9013-838180b406a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:50.272005 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916b33b-cf6f-4d48-9013-838180b406a7 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:50.273032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916b33b-cf6f-4d48-9013-838180b406a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:50.273505 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916b33b-cf6f-4d48-9013-838180b406a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:50.274682 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916b33b-cf6f-4d48-9013-838180b406a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:50.275155 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916b33b-cf6f-4d48-9013-838180b406a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:50.276954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916b33b-cf6f-4d48-9013-838180b406a7 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:50.285043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:50.293019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 353/704] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:50.294058 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916b33b-cf6f-4d48-9013-838180b406a7 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:50.313977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25eca52-26a5-4d86-a9c7-6c3c814c8dc6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:50.322687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:50.350920 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429d3ea-db31-4e63-a93b-72c2df979a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:50.351697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429d3ea-db31-4e63-a93b-72c2df979a5e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:50.352065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429d3ea-db31-4e63-a93b-72c2df979a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:50.352339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429d3ea-db31-4e63-a93b-72c2df979a5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:50.359282 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429d3ea-db31-4e63-a93b-72c2df979a5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:50.360820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429d3ea-db31-4e63-a93b-72c2df979a5e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-e429d3ea-db31-4e63-a93b-72c2df979a5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:50.375252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:50.376139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:50.377034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:50.377034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:50.377034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:50.377034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:50.377034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:50.377034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:50.377034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:50.377034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:50.381281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 352/705] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:50.382342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429d3ea-db31-4e63-a93b-72c2df979a5e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:50.416030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916b33b-cf6f-4d48-9013-838180b406a7 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:50.455805 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce5cc8-feca-4404-b612-4180e3f45681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:50.456739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce5cc8-feca-4404-b612-4180e3f45681 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:50.457187 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce5cc8-feca-4404-b612-4180e3f45681 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:50.457622 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce5cc8-feca-4404-b612-4180e3f45681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:50.469470 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce5cc8-feca-4404-b612-4180e3f45681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:50.470949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce5cc8-feca-4404-b612-4180e3f45681 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:50.485291 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429d3ea-db31-4e63-a93b-72c2df979a5e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-43ce5cc8-feca-4404-b612-4180e3f45681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:50.485796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:50.487119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:50.489002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:50.489002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:50.489002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:50.491963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 354/706] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:50.492607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce5cc8-feca-4404-b612-4180e3f45681 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:50.524857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd263a-c34f-4097-9672-f4ae805efcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:50.525353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd263a-c34f-4097-9672-f4ae805efcdb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:50.525629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd263a-c34f-4097-9672-f4ae805efcdb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:50.525891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd263a-c34f-4097-9672-f4ae805efcdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:50.526642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd263a-c34f-4097-9672-f4ae805efcdb None admin] RBAC: Adding query filter params (role.id=2b68b618a1b846f69bbe9aa08852a66f, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=e926034945b44087b378b3468954a18a) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:50.526935 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd263a-c34f-4097-9672-f4ae805efcdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:50.528903 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd263a-c34f-4097-9672-f4ae805efcdb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:50.534127 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 353/707] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:18:50 2022] GET /identity/v3/role_assignments?user.id=e926034945b44087b378b3468954a18a&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=2b68b618a1b846f69bbe9aa08852a66f => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:50.534826 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd263a-c34f-4097-9672-f4ae805efcdb None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:50.537526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:50.603438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:50.643772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce5cc8-feca-4404-b612-4180e3f45681 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:50.659055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd263a-c34f-4097-9672-f4ae805efcdb None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:50.682975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d764766-5940-4f6b-832b-fd5d5b1929b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:50.683843 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d764766-5940-4f6b-832b-fd5d5b1929b7 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:50.684225 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d764766-5940-4f6b-832b-fd5d5b1929b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:50.684575 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d764766-5940-4f6b-832b-fd5d5b1929b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:50.685413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d764766-5940-4f6b-832b-fd5d5b1929b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:50.685935 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d764766-5940-4f6b-832b-fd5d5b1929b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:50.687491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d764766-5940-4f6b-832b-fd5d5b1929b7 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:50.707605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7542b-1d28-4676-b716-abc249b339f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:50.708177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 355/708] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:50.708910 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d764766-5940-4f6b-832b-fd5d5b1929b7 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:50.709212 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7542b-1d28-4676-b716-abc249b339f5 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:50.709725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7542b-1d28-4676-b716-abc249b339f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:50.710132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7542b-1d28-4676-b716-abc249b339f5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:50.710888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7542b-1d28-4676-b716-abc249b339f5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:50.712814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7542b-1d28-4676-b716-abc249b339f5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-9ea7542b-1d28-4676-b716-abc249b339f5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:50.726536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:50.727812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:50.728858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 354/709] 10.222.0.23 () {56 vars in 1129 bytes} [Fri Sep 9 16:18:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:50.729809 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7542b-1d28-4676-b716-abc249b339f5 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:50.808926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:50.829369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d764766-5940-4f6b-832b-fd5d5b1929b7 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:50.846690 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7542b-1d28-4676-b716-abc249b339f5 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:50.863470 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f53792-152c-4071-b47c-1d829e82e345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:50.864083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f53792-152c-4071-b47c-1d829e82e345 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:50.864384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f53792-152c-4071-b47c-1d829e82e345 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:50.864615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f53792-152c-4071-b47c-1d829e82e345 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:50.865279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f53792-152c-4071-b47c-1d829e82e345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:50.865664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f53792-152c-4071-b47c-1d829e82e345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:50.867104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f53792-152c-4071-b47c-1d829e82e345 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:50.869815 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:50.882639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 356/710] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:50.883684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6dc605-a2ac-4f19-bf19-a910ee8c1f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:50.884403 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6dc605-a2ac-4f19-bf19-a910ee8c1f2e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:50.884614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce321b9-1368-4f7b-b766-2c91a6b5419d None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:50.884943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6dc605-a2ac-4f19-bf19-a910ee8c1f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:50.885218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce321b9-1368-4f7b-b766-2c91a6b5419d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:50.885346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6dc605-a2ac-4f19-bf19-a910ee8c1f2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:50.885607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce321b9-1368-4f7b-b766-2c91a6b5419d None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:50.886258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6dc605-a2ac-4f19-bf19-a910ee8c1f2e None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=152a41ad598b4b01b1881d928c53c848) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:50.886690 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6dc605-a2ac-4f19-bf19-a910ee8c1f2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:50.887022 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 357/711] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:50.887571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce321b9-1368-4f7b-b766-2c91a6b5419d None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:50.892126 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6dc605-a2ac-4f19-bf19-a910ee8c1f2e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:50.898190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 355/712] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:18:50 2022] GET /identity/v3/role_assignments?user.id=152a41ad598b4b01b1881d928c53c848&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=c6da7ba2be504943af29338ee8855f05 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:50.899145 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6dc605-a2ac-4f19-bf19-a910ee8c1f2e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:51.018462 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce321b9-1368-4f7b-b766-2c91a6b5419d None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:51.025143 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6dc605-a2ac-4f19-bf19-a910ee8c1f2e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:51.056724 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d75a7-061b-49dc-8efb-38521ac75760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:51.057740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d75a7-061b-49dc-8efb-38521ac75760 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:51.058117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d75a7-061b-49dc-8efb-38521ac75760 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:51.059012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d75a7-061b-49dc-8efb-38521ac75760 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:51.061321 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d75a7-061b-49dc-8efb-38521ac75760 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:51.061749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d75a7-061b-49dc-8efb-38521ac75760 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:51.063298 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d75a7-061b-49dc-8efb-38521ac75760 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:51.065400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:51.071531 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 358/713] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Sep 9 16:18:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:51.072987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9009e-84aa-47b3-af80-5526822e83e9 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:51.073450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9009e-84aa-47b3-af80-5526822e83e9 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:51.073801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9009e-84aa-47b3-af80-5526822e83e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:51.075933 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a0b85-8762-455b-8409-59e69fd285b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:51.076769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a0b85-8762-455b-8409-59e69fd285b8 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:51.077300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a0b85-8762-455b-8409-59e69fd285b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:51.077818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a0b85-8762-455b-8409-59e69fd285b8 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:51.078740 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a0b85-8762-455b-8409-59e69fd285b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:51.080560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a0b85-8762-455b-8409-59e69fd285b8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:51.096186 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-fbb9009e-84aa-47b3-af80-5526822e83e9 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:51.097233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-fbb9009e-84aa-47b3-af80-5526822e83e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:51.115660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 356/714] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:18:50 2022] POST /identity/v3/users => generated 254 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:18:51.130543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a0b85-8762-455b-8409-59e69fd285b8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:51.191341 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:51.232539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 359/715] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:51.270378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a0b85-8762-455b-8409-59e69fd285b8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:51.313637 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5367f-e799-41d5-bb4c-ed05fc070ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:51.314197 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5367f-e799-41d5-bb4c-ed05fc070ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:51.314472 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5367f-e799-41d5-bb4c-ed05fc070ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:51.314776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5367f-e799-41d5-bb4c-ed05fc070ff4 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:51.315500 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5367f-e799-41d5-bb4c-ed05fc070ff4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:51.316240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5367f-e799-41d5-bb4c-ed05fc070ff4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:51.317814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5367f-e799-41d5-bb4c-ed05fc070ff4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:51.324703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 357/716] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Sep 9 16:18:51 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:51.597785 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9009e-84aa-47b3-af80-5526822e83e9 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:51.703069 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9009e-84aa-47b3-af80-5526822e83e9 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:51.744442 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e823b-73c2-4a54-bd08-adbf928e4961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:51.745137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e823b-73c2-4a54-bd08-adbf928e4961 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:51.745417 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e823b-73c2-4a54-bd08-adbf928e4961 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:51.745686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e823b-73c2-4a54-bd08-adbf928e4961 None admin] PATH_INFO: `/v3/services/network` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:51.746282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e823b-73c2-4a54-bd08-adbf928e4961 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:51.747682 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e823b-73c2-4a54-bd08-adbf928e4961 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-c78e823b-73c2-4a54-bd08-adbf928e4961 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:51.759046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:51.760028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:51.760850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 360/717] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:51 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:51.766129 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5367f-e799-41d5-bb4c-ed05fc070ff4 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:51.885415 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:51.948428 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5367f-e799-41d5-bb4c-ed05fc070ff4 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:51.999047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc680b-45f3-4771-975d-41a20a7174b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:52.000043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc680b-45f3-4771-975d-41a20a7174b5 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:52.000509 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc680b-45f3-4771-975d-41a20a7174b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:52.000981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc680b-45f3-4771-975d-41a20a7174b5 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:52.001812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc680b-45f3-4771-975d-41a20a7174b5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:52.002299 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc680b-45f3-4771-975d-41a20a7174b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:52.003977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc680b-45f3-4771-975d-41a20a7174b5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:52.017604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 358/718] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Sep 9 16:18:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:52.023007 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e823b-73c2-4a54-bd08-adbf928e4961 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:52.122427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:52.181825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e823b-73c2-4a54-bd08-adbf928e4961 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:52.219749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ce936-5cfe-4173-90c7-cb30151103cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:52.220371 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ce936-5cfe-4173-90c7-cb30151103cb None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:52.220650 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ce936-5cfe-4173-90c7-cb30151103cb None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:52.220921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ce936-5cfe-4173-90c7-cb30151103cb None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:52.221556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ce936-5cfe-4173-90c7-cb30151103cb None admin] RBAC: Adding query filter params (type=network) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:52.221857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ce936-5cfe-4173-90c7-cb30151103cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:52.223267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ce936-5cfe-4173-90c7-cb30151103cb None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:52.235112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 361/719] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Sep 9 16:18:52 2022] GET /identity/v3/services?type=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:53.199058 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946426ec-79e7-48ea-b112-411de59e63b7 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:53.199586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946426ec-79e7-48ea-b112-411de59e63b7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:53.199586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946426ec-79e7-48ea-b112-411de59e63b7 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:53.200928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 359/720] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:53.206341 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735cf9-d3bd-4989-8db3-944b209b507d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:53.206778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735cf9-d3bd-4989-8db3-944b209b507d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:53.206947 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735cf9-d3bd-4989-8db3-944b209b507d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:53.230818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-cf735cf9-d3bd-4989-8db3-944b209b507d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:53.232004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cf735cf9-d3bd-4989-8db3-944b209b507d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:53.304835 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 362/721] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:53.504734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de7053-0f3c-4b8d-8ac5-89f0796a4234 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:53.505433 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de7053-0f3c-4b8d-8ac5-89f0796a4234 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:53.506067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de7053-0f3c-4b8d-8ac5-89f0796a4234 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:53.507886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 360/722] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:53.513312 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3a41e-0e96-41b7-8fb9-7ab6c348d52d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:53.513875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3a41e-0e96-41b7-8fb9-7ab6c348d52d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:53.514338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3a41e-0e96-41b7-8fb9-7ab6c348d52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:53.532739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-8fd3a41e-0e96-41b7-8fb9-7ab6c348d52d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:53.534217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8fd3a41e-0e96-41b7-8fb9-7ab6c348d52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:53.580951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:53.604069 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 363/723] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:53.731423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de7053-0f3c-4b8d-8ac5-89f0796a4234 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:53.860012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de7053-0f3c-4b8d-8ac5-89f0796a4234 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:53.881117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:53.896790 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3a41e-0e96-41b7-8fb9-7ab6c348d52d None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:53.910828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1976c-3e6f-464d-8701-7fd037b26770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:53.911429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1976c-3e6f-464d-8701-7fd037b26770 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:53.911710 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1976c-3e6f-464d-8701-7fd037b26770 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:53.911983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1976c-3e6f-464d-8701-7fd037b26770 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:53.935805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1976c-3e6f-464d-8701-7fd037b26770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-a4a1976c-3e6f-464d-8701-7fd037b26770 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:53.940238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:53.941524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:53.942331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:53.942331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:53.942331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:53.942331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:53.942331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:53.942331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:53.942331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:53.942331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:53.942331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:53.942331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:53.942331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:18:53.942331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:53.946715 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 361/724] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:18:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:53.947883 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1976c-3e6f-464d-8701-7fd037b26770 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:54.010278 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3a41e-0e96-41b7-8fb9-7ab6c348d52d None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:54.045237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a5673-9a6f-4d34-9132-5bbe377eb462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:54.045880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a5673-9a6f-4d34-9132-5bbe377eb462 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:54.046158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a5673-9a6f-4d34-9132-5bbe377eb462 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:54.046450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a5673-9a6f-4d34-9132-5bbe377eb462 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:54.056574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a5673-9a6f-4d34-9132-5bbe377eb462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-689a5673-9a6f-4d34-9132-5bbe377eb462 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:54.061631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:54.066883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:54.067799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:54.067799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:54.067799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:54.067799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:54.067799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:54.067799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:54.067799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:54.067799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:54.067799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:54.067799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:54.067799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:18:54.067799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:54.067799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 364/725] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:18:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:54.067799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7869-9d1e-4011-a690-5737a850328d None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:54.068590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7869-9d1e-4011-a690-5737a850328d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:54.068590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7869-9d1e-4011-a690-5737a850328d None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:54.068590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 365/726] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:54.068590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab868e-b4d5-41c1-be7a-34864525f2ca None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:54.068590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab868e-b4d5-41c1-be7a-34864525f2ca None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:54.068590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab868e-b4d5-41c1-be7a-34864525f2ca None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:54.070657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 366/727] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:54.071448 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab868e-b4d5-41c1-be7a-34864525f2ca None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:54.089803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1976c-3e6f-464d-8701-7fd037b26770 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:54.116288 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:54.125138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a41c5b-0c5f-41f2-ad69-f4453a7374f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:54.125745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a41c5b-0c5f-41f2-ad69-f4453a7374f1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:54.126096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a41c5b-0c5f-41f2-ad69-f4453a7374f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:54.126436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a41c5b-0c5f-41f2-ad69-f4453a7374f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:54.127198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a41c5b-0c5f-41f2-ad69-f4453a7374f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:54.127569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a41c5b-0c5f-41f2-ad69-f4453a7374f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:54.128939 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a41c5b-0c5f-41f2-ad69-f4453a7374f1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:54.141735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 362/728] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:54.141735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115c57b-d440-40cd-81b3-5e734fef628f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:54.141735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115c57b-d440-40cd-81b3-5e734fef628f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:54.141735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115c57b-d440-40cd-81b3-5e734fef628f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:54.156795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d115c57b-d440-40cd-81b3-5e734fef628f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:54.157973 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d115c57b-d440-40cd-81b3-5e734fef628f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:54.186110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab868e-b4d5-41c1-be7a-34864525f2ca None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:54.186750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:54.221616 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c02aa8-d885-4387-b6a3-9a4eb4f20db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:54.222354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c02aa8-d885-4387-b6a3-9a4eb4f20db6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:54.222487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c02aa8-d885-4387-b6a3-9a4eb4f20db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:54.222792 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c02aa8-d885-4387-b6a3-9a4eb4f20db6 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:54.223491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c02aa8-d885-4387-b6a3-9a4eb4f20db6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:54.223751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c02aa8-d885-4387-b6a3-9a4eb4f20db6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:54.225402 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c02aa8-d885-4387-b6a3-9a4eb4f20db6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:54.234668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 363/729] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:54.234668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16fa2c-7883-4b3f-8bbd-c42b6818f0a9 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:54.234668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16fa2c-7883-4b3f-8bbd-c42b6818f0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:54.234668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16fa2c-7883-4b3f-8bbd-c42b6818f0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:54.244663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 367/730] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:18:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:54.245700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c02aa8-d885-4387-b6a3-9a4eb4f20db6 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:54.248588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3d16fa2c-7883-4b3f-8bbd-c42b6818f0a9 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:54.249826 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3d16fa2c-7883-4b3f-8bbd-c42b6818f0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:54.334651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 364/731] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:54.336247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16fa2c-7883-4b3f-8bbd-c42b6818f0a9 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:54.368744 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:54.371544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c02aa8-d885-4387-b6a3-9a4eb4f20db6 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:54.410264 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db49df-18a0-440f-865f-826a7240c01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:54.410929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db49df-18a0-440f-865f-826a7240c01c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:54.411209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db49df-18a0-440f-865f-826a7240c01c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:54.411480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db49df-18a0-440f-865f-826a7240c01c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:54.423347 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db49df-18a0-440f-865f-826a7240c01c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:54.424929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db49df-18a0-440f-865f-826a7240c01c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-a0db49df-18a0-440f-865f-826a7240c01c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:54.440153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:54.441512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:54.442396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:54.442396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:54.442396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:54.442396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 368/732] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:54.448977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db49df-18a0-440f-865f-826a7240c01c None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:54.543026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:54.549983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16fa2c-7883-4b3f-8bbd-c42b6818f0a9 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:54.585636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0cac2-88fc-4978-84c7-2d0a769b95b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:54.586263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0cac2-88fc-4978-84c7-2d0a769b95b8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:54.586561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0cac2-88fc-4978-84c7-2d0a769b95b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:54.586837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0cac2-88fc-4978-84c7-2d0a769b95b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:54.592487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db49df-18a0-440f-865f-826a7240c01c None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:54.597333 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0cac2-88fc-4978-84c7-2d0a769b95b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:54.598837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0cac2-88fc-4978-84c7-2d0a769b95b8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-30e0cac2-88fc-4978-84c7-2d0a769b95b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:54.613918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:54.614963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:54.616017 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:54.616017 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:54.616017 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:54.618621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 365/733] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:54.619564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0cac2-88fc-4978-84c7-2d0a769b95b8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:54.632728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a662167-30c1-4e1e-91ff-5e4733b51703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:54.633724 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a662167-30c1-4e1e-91ff-5e4733b51703 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:54.634189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a662167-30c1-4e1e-91ff-5e4733b51703 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:54.634695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a662167-30c1-4e1e-91ff-5e4733b51703 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:54.635556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a662167-30c1-4e1e-91ff-5e4733b51703 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:54.636554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a662167-30c1-4e1e-91ff-5e4733b51703 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:54.638848 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a662167-30c1-4e1e-91ff-5e4733b51703 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:54.653404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 369/734] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:54.654476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a662167-30c1-4e1e-91ff-5e4733b51703 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:54.670618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:54.723245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0cac2-88fc-4978-84c7-2d0a769b95b8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:54.760253 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9afec-bfbf-410b-bbf0-75f63c38619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:54.760865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9afec-bfbf-410b-bbf0-75f63c38619e None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:54.761141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9afec-bfbf-410b-bbf0-75f63c38619e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:54.761422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9afec-bfbf-410b-bbf0-75f63c38619e None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:54.762078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9afec-bfbf-410b-bbf0-75f63c38619e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:54.763466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a662167-30c1-4e1e-91ff-5e4733b51703 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:54.763574 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9afec-bfbf-410b-bbf0-75f63c38619e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:54.780848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 366/735] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Sep 9 16:18:54 2022] POST /identity/v3/services => generated 251 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:18:54.781997 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9afec-bfbf-410b-bbf0-75f63c38619e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:54.798122 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:54.822742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877aa65c-c564-478c-bab4-d42639107028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:54.824021 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877aa65c-c564-478c-bab4-d42639107028 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:54.824497 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877aa65c-c564-478c-bab4-d42639107028 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:54.825126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877aa65c-c564-478c-bab4-d42639107028 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:54.826647 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877aa65c-c564-478c-bab4-d42639107028 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:54.827285 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877aa65c-c564-478c-bab4-d42639107028 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:54.828888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877aa65c-c564-478c-bab4-d42639107028 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:54.845826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 370/736] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:54.847178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877aa65c-c564-478c-bab4-d42639107028 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:54.934264 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9afec-bfbf-410b-bbf0-75f63c38619e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:54.941888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:54.982770 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a3684-0b16-4faa-8a15-03d7b64502ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:54.983415 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3684-0b16-4faa-8a15-03d7b64502ab None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:54.983695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3684-0b16-4faa-8a15-03d7b64502ab None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:54.983975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3684-0b16-4faa-8a15-03d7b64502ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:54.992538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877aa65c-c564-478c-bab4-d42639107028 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:54.997448 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a3684-0b16-4faa-8a15-03d7b64502ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-1c0a3684-0b16-4faa-8a15-03d7b64502ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:18:55.003469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:55.004421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:55.005337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:55.005337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:55.005337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:55.005337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:55.005337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:18:55.005337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:18:55.005337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:18:55.005337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:18:55.005337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:18:55.005337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:18:55.005337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:18:55.005337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:55.005337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 367/737] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:18:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:55.006147 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a3684-0b16-4faa-8a15-03d7b64502ab None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:55.036724 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d88b06-6d2f-4f6d-a9f1-92f4014c296a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:55.037441 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d88b06-6d2f-4f6d-a9f1-92f4014c296a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:55.037738 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d88b06-6d2f-4f6d-a9f1-92f4014c296a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:55.037991 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d88b06-6d2f-4f6d-a9f1-92f4014c296a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:55.045389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d88b06-6d2f-4f6d-a9f1-92f4014c296a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:55.046866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d88b06-6d2f-4f6d-a9f1-92f4014c296a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-e3d88b06-6d2f-4f6d-a9f1-92f4014c296a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:55.058469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:55.059502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:55.060454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:55.060454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:55.060454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:55.060454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:55.060454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:55.060454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:55.060454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:55.060454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:55.060454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 16:18:55.060454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:55.060454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 371/738] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:18:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:55.061028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d88b06-6d2f-4f6d-a9f1-92f4014c296a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:55.082394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:55.118962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a3684-0b16-4faa-8a15-03d7b64502ab None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:55.152882 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a36d5-45a5-430f-886f-cb24cd1a6e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:55.153518 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a36d5-45a5-430f-886f-cb24cd1a6e1b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:55.153796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a36d5-45a5-430f-886f-cb24cd1a6e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:55.154060 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a36d5-45a5-430f-886f-cb24cd1a6e1b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:55.161798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a36d5-45a5-430f-886f-cb24cd1a6e1b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:55.163375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a36d5-45a5-430f-886f-cb24cd1a6e1b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:55.166460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d88b06-6d2f-4f6d-a9f1-92f4014c296a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-9a1a36d5-45a5-430f-886f-cb24cd1a6e1b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:55.175169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:55.176307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:55.177464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:55.177464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:55.177464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:55.177464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:55.177464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:55.177464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:55.177464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:55.177464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:55.177464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:18:55.177464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:55.178394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 368/739] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Sep 9 16:18:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:55.179858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e476fa0-df98-473d-a445-302434b6df04 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:55.180138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e476fa0-df98-473d-a445-302434b6df04 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:55.180409 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e476fa0-df98-473d-a445-302434b6df04 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:55.182530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 369/740] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:55.182756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e476fa0-df98-473d-a445-302434b6df04 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:55.200068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:55.209358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacc72a-daf7-48be-9099-39993f74566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:55.210476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacc72a-daf7-48be-9099-39993f74566f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:55.210845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacc72a-daf7-48be-9099-39993f74566f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:55.211325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacc72a-daf7-48be-9099-39993f74566f None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:55.212062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacc72a-daf7-48be-9099-39993f74566f None admin] RBAC: Adding query filter params (name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:55.212598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacc72a-daf7-48be-9099-39993f74566f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:55.214386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacc72a-daf7-48be-9099-39993f74566f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:55.230223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 372/741] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:18:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:55.230223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacc72a-daf7-48be-9099-39993f74566f None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:55.344023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacc72a-daf7-48be-9099-39993f74566f None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:55.379961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dda99a-e9e5-46ed-8396-95912307c2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:55.380591 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dda99a-e9e5-46ed-8396-95912307c2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:55.380919 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dda99a-e9e5-46ed-8396-95912307c2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:55.381184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dda99a-e9e5-46ed-8396-95912307c2f1 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:55.381920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dda99a-e9e5-46ed-8396-95912307c2f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:55.382223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dda99a-e9e5-46ed-8396-95912307c2f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:55.383775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dda99a-e9e5-46ed-8396-95912307c2f1 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:55.405218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 373/742] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Sep 9 16:18:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:55.406830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28081d65-a1b6-40e7-be98-168331031d17 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:55.407177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28081d65-a1b6-40e7-be98-168331031d17 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:55.407512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28081d65-a1b6-40e7-be98-168331031d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:55.423476 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e476fa0-df98-473d-a445-302434b6df04 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:55.425627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-28081d65-a1b6-40e7-be98-168331031d17 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:55.426848 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-28081d65-a1b6-40e7-be98-168331031d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:55.452016 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:55.460859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c7e84-7b3d-423b-8200-213093177909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:55.461467 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c7e84-7b3d-423b-8200-213093177909 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:55.461763 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c7e84-7b3d-423b-8200-213093177909 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:55.462005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c7e84-7b3d-423b-8200-213093177909 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:55.462745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c7e84-7b3d-423b-8200-213093177909 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:55.463140 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c7e84-7b3d-423b-8200-213093177909 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:55.464694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c7e84-7b3d-423b-8200-213093177909 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:55.473212 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:55.483680 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 370/743] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Sep 9 16:18:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:55.484591 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c7e84-7b3d-423b-8200-213093177909 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:55.510534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 374/744] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 16:18:55.510534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28081d65-a1b6-40e7-be98-168331031d17 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:55.616686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c7e84-7b3d-423b-8200-213093177909 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:55.635106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28081d65-a1b6-40e7-be98-168331031d17 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:55.663052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7948a1-612d-4694-87d8-9562e0ae24a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:55.664688 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7948a1-612d-4694-87d8-9562e0ae24a7 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:55.665319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7948a1-612d-4694-87d8-9562e0ae24a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:55.665851 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7948a1-612d-4694-87d8-9562e0ae24a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:55.672824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d11b4-236f-45e3-8c94-9185aaa44c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:55.673565 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d11b4-236f-45e3-8c94-9185aaa44c00 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:55.674381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d11b4-236f-45e3-8c94-9185aaa44c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:55.674778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d11b4-236f-45e3-8c94-9185aaa44c00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:55.678635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7948a1-612d-4694-87d8-9562e0ae24a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:55.680249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7948a1-612d-4694-87d8-9562e0ae24a7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:55.686365 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d11b4-236f-45e3-8c94-9185aaa44c00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:55.687959 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d11b4-236f-45e3-8c94-9185aaa44c00 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-aa7948a1-612d-4694-87d8-9562e0ae24a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:55.699209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:55.700398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:55.701376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:55.701376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:55.701376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:55.703011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 371/745] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:55.704164 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7948a1-612d-4694-87d8-9562e0ae24a7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-f75d11b4-236f-45e3-8c94-9185aaa44c00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:55.705660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:55.706782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:55.707922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:55.707922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:55.707922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:55.710581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 375/746] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:55.711829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d11b4-236f-45e3-8c94-9185aaa44c00 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:55.754195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:55.759201 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:55.821804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7948a1-612d-4694-87d8-9562e0ae24a7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:55.831183 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d11b4-236f-45e3-8c94-9185aaa44c00 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:55.855557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bf95a-0f92-4bf3-a718-5c0c997e39c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:55.856167 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bf95a-0f92-4bf3-a718-5c0c997e39c2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:55.856441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bf95a-0f92-4bf3-a718-5c0c997e39c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:55.856707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bf95a-0f92-4bf3-a718-5c0c997e39c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:55.870154 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c818bb-fc3a-4cec-bd1f-49eacefca0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:55.871261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c818bb-fc3a-4cec-bd1f-49eacefca0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:55.871741 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c818bb-fc3a-4cec-bd1f-49eacefca0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:55.872667 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c818bb-fc3a-4cec-bd1f-49eacefca0c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:55.873762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c818bb-fc3a-4cec-bd1f-49eacefca0c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:55.874263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c818bb-fc3a-4cec-bd1f-49eacefca0c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:55.875981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c818bb-fc3a-4cec-bd1f-49eacefca0c9 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:55.890225 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 376/747] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:55.891378 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c818bb-fc3a-4cec-bd1f-49eacefca0c9 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:55.932529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bf95a-0f92-4bf3-a718-5c0c997e39c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:55.933986 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bf95a-0f92-4bf3-a718-5c0c997e39c2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:55.973854 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:55.996535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:56.006659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c818bb-fc3a-4cec-bd1f-49eacefca0c9 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:56.037365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 372/748] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:55 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 16:18:56.038210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bf95a-0f92-4bf3-a718-5c0c997e39c2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:56.046018 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98e55e-f2b6-4cfd-9a84-485c4f3e8950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:56.046680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98e55e-f2b6-4cfd-9a84-485c4f3e8950 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:56.046958 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98e55e-f2b6-4cfd-9a84-485c4f3e8950 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:56.047229 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98e55e-f2b6-4cfd-9a84-485c4f3e8950 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:56.047877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98e55e-f2b6-4cfd-9a84-485c4f3e8950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:56.048180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98e55e-f2b6-4cfd-9a84-485c4f3e8950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:56.049575 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98e55e-f2b6-4cfd-9a84-485c4f3e8950 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:56.062808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 377/749] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:56.063705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98e55e-f2b6-4cfd-9a84-485c4f3e8950 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:56.156141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bf95a-0f92-4bf3-a718-5c0c997e39c2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:56.177333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98e55e-f2b6-4cfd-9a84-485c4f3e8950 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:56.189547 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9627f-f8f1-4f7f-a00f-e8132bf2f94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:56.190300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9627f-f8f1-4f7f-a00f-e8132bf2f94d None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:56.190980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9627f-f8f1-4f7f-a00f-e8132bf2f94d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:56.191301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9627f-f8f1-4f7f-a00f-e8132bf2f94d None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:56.192376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9627f-f8f1-4f7f-a00f-e8132bf2f94d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:56.193806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9627f-f8f1-4f7f-a00f-e8132bf2f94d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:56.205884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:56.212128 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 373/750] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Sep 9 16:18:56 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:18:56.213155 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9627f-f8f1-4f7f-a00f-e8132bf2f94d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:56.233164 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437b8f3-f5cc-43c8-997e-556a5fc44098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:56.233928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437b8f3-f5cc-43c8-997e-556a5fc44098 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:56.234302 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437b8f3-f5cc-43c8-997e-556a5fc44098 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:56.234755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437b8f3-f5cc-43c8-997e-556a5fc44098 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:56.248188 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437b8f3-f5cc-43c8-997e-556a5fc44098 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:56.249865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437b8f3-f5cc-43c8-997e-556a5fc44098 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-2437b8f3-f5cc-43c8-997e-556a5fc44098 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:18:56.264420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:56.265676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:56.266865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:56.266865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:56.266865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:18:56.266865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:18:56.266865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:18:56.266865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:18:56.266865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:18:56.266865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:18:56.266865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 16:18:56.266865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:56.273017 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 378/751] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Sep 9 16:18:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:56.273017 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437b8f3-f5cc-43c8-997e-556a5fc44098 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:56.282601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:56.349274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9627f-f8f1-4f7f-a00f-e8132bf2f94d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:56.389665 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437b8f3-f5cc-43c8-997e-556a5fc44098 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:56.391240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd2a40-8079-4d6e-9ece-d43d60255334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:56.392357 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2a40-8079-4d6e-9ece-d43d60255334 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:56.393844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2a40-8079-4d6e-9ece-d43d60255334 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:56.394326 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2a40-8079-4d6e-9ece-d43d60255334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:56.407702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd2a40-8079-4d6e-9ece-d43d60255334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:56.409452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd2a40-8079-4d6e-9ece-d43d60255334 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-5dfd2a40-8079-4d6e-9ece-d43d60255334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:56.425396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:56.426947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:56.427966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:56.427966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:56.427966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:56.431632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067405b-7840-4a98-a1be-5411af484cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:56.431788 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 374/752] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:56.431976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067405b-7840-4a98-a1be-5411af484cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:56.432255 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067405b-7840-4a98-a1be-5411af484cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:56.432524 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067405b-7840-4a98-a1be-5411af484cd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:56.432625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd2a40-8079-4d6e-9ece-d43d60255334 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:56.443316 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067405b-7840-4a98-a1be-5411af484cd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:56.444807 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067405b-7840-4a98-a1be-5411af484cd3 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-5067405b-7840-4a98-a1be-5411af484cd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:56.460950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:56.462088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:56.463253 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:56.463253 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:56.463253 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:56.463253 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:56.463253 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:56.463253 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:56.463253 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:56.463253 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:56.466720 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 379/753] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:56.466720 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067405b-7840-4a98-a1be-5411af484cd3 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:56.503442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:56.534221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:56.569003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd2a40-8079-4d6e-9ece-d43d60255334 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:56.582767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067405b-7840-4a98-a1be-5411af484cd3 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:56.616301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aff7d6-0b0c-452e-9a3c-ed14f192d9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:56.616927 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aff7d6-0b0c-452e-9a3c-ed14f192d9ac None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:56.617204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aff7d6-0b0c-452e-9a3c-ed14f192d9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:56.617470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aff7d6-0b0c-452e-9a3c-ed14f192d9ac None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:56.618198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aff7d6-0b0c-452e-9a3c-ed14f192d9ac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:56.618523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aff7d6-0b0c-452e-9a3c-ed14f192d9ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:56.619970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aff7d6-0b0c-452e-9a3c-ed14f192d9ac None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:56.628190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff907d-ecb4-487d-9a91-56936b190ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:56.629459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff907d-ecb4-487d-9a91-56936b190ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:56.629938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff907d-ecb4-487d-9a91-56936b190ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:56.630402 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff907d-ecb4-487d-9a91-56936b190ca8 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:56.632585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff907d-ecb4-487d-9a91-56936b190ca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:56.633110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff907d-ecb4-487d-9a91-56936b190ca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:56.634754 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff907d-ecb4-487d-9a91-56936b190ca8 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:56.645003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 375/754] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Sep 9 16:18:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:56.645925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aff7d6-0b0c-452e-9a3c-ed14f192d9ac None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:56.650990 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 380/755] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:56.651663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff907d-ecb4-487d-9a91-56936b190ca8 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:56.749160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aff7d6-0b0c-452e-9a3c-ed14f192d9ac None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:56.759416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:56.785798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78ed65-388c-49eb-b23c-f53668d449e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:56.786430 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78ed65-388c-49eb-b23c-f53668d449e8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:56.786751 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78ed65-388c-49eb-b23c-f53668d449e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:56.787093 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78ed65-388c-49eb-b23c-f53668d449e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:56.787794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78ed65-388c-49eb-b23c-f53668d449e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:56.788098 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78ed65-388c-49eb-b23c-f53668d449e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:56.789545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78ed65-388c-49eb-b23c-f53668d449e8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:56.790149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff907d-ecb4-487d-9a91-56936b190ca8 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:56.804144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 376/756] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:56.805321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78ed65-388c-49eb-b23c-f53668d449e8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:56.825097 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:56.838348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f13a10-6eb5-4492-ac9c-15855f33ef55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:56.839757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f13a10-6eb5-4492-ac9c-15855f33ef55 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:56.840458 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f13a10-6eb5-4492-ac9c-15855f33ef55 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:56.840934 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f13a10-6eb5-4492-ac9c-15855f33ef55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:56.924085 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f13a10-6eb5-4492-ac9c-15855f33ef55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:56.926735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f13a10-6eb5-4492-ac9c-15855f33ef55 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:56.930935 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78ed65-388c-49eb-b23c-f53668d449e8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:56.970125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148484a5-746a-4a52-b70b-c6d5d3238c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:56.970924 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148484a5-746a-4a52-b70b-c6d5d3238c18 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:56.971350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148484a5-746a-4a52-b70b-c6d5d3238c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:56.971500 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148484a5-746a-4a52-b70b-c6d5d3238c18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:56.978699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148484a5-746a-4a52-b70b-c6d5d3238c18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:56.980242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148484a5-746a-4a52-b70b-c6d5d3238c18 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-148484a5-746a-4a52-b70b-c6d5d3238c18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:56.991979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:56.993104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:56.994002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:56.994002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:56.994002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:56.994002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:56.994002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:56.994002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:56.994002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:56.994002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:56.994002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 377/757] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:56.994545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148484a5-746a-4a52-b70b-c6d5d3238c18 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:57.026877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:57.058172 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 381/758] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:18:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:18:57.059875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4e1b2-081b-4956-a99e-495b1c917955 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:57.060866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4e1b2-081b-4956-a99e-495b1c917955 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:57.061386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4e1b2-081b-4956-a99e-495b1c917955 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:57.063914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 382/759] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:57.064616 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4e1b2-081b-4956-a99e-495b1c917955 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:57.093800 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:57.109084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148484a5-746a-4a52-b70b-c6d5d3238c18 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:57.142422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5d780-cf99-4526-bd8e-299008b6b6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:57.143105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5d780-cf99-4526-bd8e-299008b6b6ff None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:57.143377 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5d780-cf99-4526-bd8e-299008b6b6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:57.143639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5d780-cf99-4526-bd8e-299008b6b6ff None admin] PATH_INFO: `/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/152a41ad598b4b01b1881d928c53c848/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:57.145549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5d780-cf99-4526-bd8e-299008b6b6ff None admin] RBAC: Authorizing `identity:create_grant(project_id=253ff30a56444df28d44c4b7eaadb276, user_id=152a41ad598b4b01b1881d928c53c848, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:57.147122 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5d780-cf99-4526-bd8e-299008b6b6ff None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:57.170531 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 378/760] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:18:56 2022] PUT /identity/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/152a41ad598b4b01b1881d928c53c848/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:18:57.170531 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5d780-cf99-4526-bd8e-299008b6b6ff None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:57.175332 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4e1b2-081b-4956-a99e-495b1c917955 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:57.222167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b57f2c-d7c0-4685-a10e-c054bf2239df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:57.222951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b57f2c-d7c0-4685-a10e-c054bf2239df None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:57.223336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b57f2c-d7c0-4685-a10e-c054bf2239df None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:57.223672 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b57f2c-d7c0-4685-a10e-c054bf2239df None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:57.224748 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b57f2c-d7c0-4685-a10e-c054bf2239df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:57.225179 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b57f2c-d7c0-4685-a10e-c054bf2239df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:57.226767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b57f2c-d7c0-4685-a10e-c054bf2239df None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:57.241364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 383/761] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:57.242832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea06551-129f-48f4-90b2-a1b1fc20d93d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:57.243117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea06551-129f-48f4-90b2-a1b1fc20d93d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:57.243442 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea06551-129f-48f4-90b2-a1b1fc20d93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:57.265080 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-0ea06551-129f-48f4-90b2-a1b1fc20d93d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:57.266471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0ea06551-129f-48f4-90b2-a1b1fc20d93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:57.293243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:57.313833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5d780-cf99-4526-bd8e-299008b6b6ff None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:57.352322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 384/762] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:18:57.353678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea06551-129f-48f4-90b2-a1b1fc20d93d None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:57.361823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2cbef-d42d-4bc3-807f-ee86feccd1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:57.362463 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cbef-d42d-4bc3-807f-ee86feccd1ab None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:57.363300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cbef-d42d-4bc3-807f-ee86feccd1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:57.363580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cbef-d42d-4bc3-807f-ee86feccd1ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:57.374088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:57.381511 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2cbef-d42d-4bc3-807f-ee86feccd1ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:57.383247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2cbef-d42d-4bc3-807f-ee86feccd1ab None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-97e2cbef-d42d-4bc3-807f-ee86feccd1ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:18:57.404817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:57.406056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:18:57.407168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:18:57.407168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:18:57.407168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:57.408943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 379/763] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:57.415004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2cbef-d42d-4bc3-807f-ee86feccd1ab None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:57.487190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea06551-129f-48f4-90b2-a1b1fc20d93d None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:57.522281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480ed8d-97d0-46b9-9d84-6b21eb07426d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:57.522931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480ed8d-97d0-46b9-9d84-6b21eb07426d None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:57.523214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480ed8d-97d0-46b9-9d84-6b21eb07426d None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:57.523488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480ed8d-97d0-46b9-9d84-6b21eb07426d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:57.524220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480ed8d-97d0-46b9-9d84-6b21eb07426d None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=e926034945b44087b378b3468954a18a) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:57.524532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480ed8d-97d0-46b9-9d84-6b21eb07426d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:57.525966 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480ed8d-97d0-46b9-9d84-6b21eb07426d None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:57.532049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 385/764] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:18:57 2022] GET /identity/v3/role_assignments?user.id=e926034945b44087b378b3468954a18a&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=c6da7ba2be504943af29338ee8855f05 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:57.613131 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2cbef-d42d-4bc3-807f-ee86feccd1ab None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:57.660664 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f619851-d520-4f0d-a70d-3b4735f115e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:57.661281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f619851-d520-4f0d-a70d-3b4735f115e9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:57.661559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f619851-d520-4f0d-a70d-3b4735f115e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:57.661826 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f619851-d520-4f0d-a70d-3b4735f115e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:57.662707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f619851-d520-4f0d-a70d-3b4735f115e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:57.663009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f619851-d520-4f0d-a70d-3b4735f115e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:57.664400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f619851-d520-4f0d-a70d-3b4735f115e9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:57.673038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480ed8d-97d0-46b9-9d84-6b21eb07426d None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:57.674289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:57.678584 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 380/765] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:57.687633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f619851-d520-4f0d-a70d-3b4735f115e9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:57.822567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:57.844086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480ed8d-97d0-46b9-9d84-6b21eb07426d None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:57.858749 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f619851-d520-4f0d-a70d-3b4735f115e9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:57.912925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd124b4-b9a8-41f7-baa3-594e575cea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:57.913470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd124b4-b9a8-41f7-baa3-594e575cea7c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:57.913747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd124b4-b9a8-41f7-baa3-594e575cea7c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:57.914010 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd124b4-b9a8-41f7-baa3-594e575cea7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:57.917476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466059a-2d99-49a2-8830-73f991cdf351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:57.918114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466059a-2d99-49a2-8830-73f991cdf351 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:57.918387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466059a-2d99-49a2-8830-73f991cdf351 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:57.918678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466059a-2d99-49a2-8830-73f991cdf351 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:57.919318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466059a-2d99-49a2-8830-73f991cdf351 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:57.920682 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466059a-2d99-49a2-8830-73f991cdf351 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:57.928284 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd124b4-b9a8-41f7-baa3-594e575cea7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:57.930595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd124b4-b9a8-41f7-baa3-594e575cea7c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-5466059a-2d99-49a2-8830-73f991cdf351 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:57.936569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:57.937557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:57.939141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 386/766] 10.222.0.23 () {56 vars in 1129 bytes} [Fri Sep 9 16:18:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:18:57.944914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466059a-2d99-49a2-8830-73f991cdf351 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-2fd124b4-b9a8-41f7-baa3-594e575cea7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:18:57.953994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:57.957204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:57.959422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:18:57.959422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:18:57.959422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:18:57.959422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:18:57.959422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:18:57.959422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:18:57.959422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:18:57.959422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:18:57.959422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 381/767] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:57.963971 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd124b4-b9a8-41f7-baa3-594e575cea7c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:58.095142 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:58.128798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466059a-2d99-49a2-8830-73f991cdf351 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:58.162403 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd124b4-b9a8-41f7-baa3-594e575cea7c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:58.191354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36197d59-d728-4397-8907-a6205fe26082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:58.193783 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36197d59-d728-4397-8907-a6205fe26082 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:58.194061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36197d59-d728-4397-8907-a6205fe26082 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:58.194333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36197d59-d728-4397-8907-a6205fe26082 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:58.195005 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36197d59-d728-4397-8907-a6205fe26082 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:58.195306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36197d59-d728-4397-8907-a6205fe26082 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:58.196818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36197d59-d728-4397-8907-a6205fe26082 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:58.216496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 387/768] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Sep 9 16:18:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:58.217076 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e1c1d-30cf-44fa-913c-24206a574160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:58.217759 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e1c1d-30cf-44fa-913c-24206a574160 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:58.218084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e1c1d-30cf-44fa-913c-24206a574160 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:58.218366 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e1c1d-30cf-44fa-913c-24206a574160 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:58.219106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e1c1d-30cf-44fa-913c-24206a574160 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:58.219413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e1c1d-30cf-44fa-913c-24206a574160 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:58.221119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e1c1d-30cf-44fa-913c-24206a574160 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:58.222012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36197d59-d728-4397-8907-a6205fe26082 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:58.239816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 382/769] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:18:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:58.244497 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:58.245288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e1c1d-30cf-44fa-913c-24206a574160 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:58.413374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36197d59-d728-4397-8907-a6205fe26082 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:58.426592 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e1c1d-30cf-44fa-913c-24206a574160 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:58.462929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:58.479172 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91726b-9b92-4a4c-ac17-70663c33ab2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:58.479802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91726b-9b92-4a4c-ac17-70663c33ab2b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:58.480081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91726b-9b92-4a4c-ac17-70663c33ab2b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:58.480353 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91726b-9b92-4a4c-ac17-70663c33ab2b None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:58.481006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91726b-9b92-4a4c-ac17-70663c33ab2b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:58.481310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91726b-9b92-4a4c-ac17-70663c33ab2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:58.482770 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91726b-9b92-4a4c-ac17-70663c33ab2b None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:58.495007 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658910e2-db11-4ce0-9adb-15751fd496f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:58.495633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658910e2-db11-4ce0-9adb-15751fd496f1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:58.495909 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658910e2-db11-4ce0-9adb-15751fd496f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:58.496176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658910e2-db11-4ce0-9adb-15751fd496f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:58.496909 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658910e2-db11-4ce0-9adb-15751fd496f1 None admin] RBAC: Adding query filter params (role.id=2b68b618a1b846f69bbe9aa08852a66f, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=048b4abace244e48b6897daec0277e2f) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:58.497213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658910e2-db11-4ce0-9adb-15751fd496f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:58.498695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658910e2-db11-4ce0-9adb-15751fd496f1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:58.500673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 388/770] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Sep 9 16:18:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:18:58.501713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91726b-9b92-4a4c-ac17-70663c33ab2b None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:58.513818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 383/771] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:18:58 2022] GET /identity/v3/role_assignments?user.id=048b4abace244e48b6897daec0277e2f&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=2b68b618a1b846f69bbe9aa08852a66f => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:58.597831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:58.633777 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91726b-9b92-4a4c-ac17-70663c33ab2b None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:58.681743 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c8442-799f-4560-ba4c-c2e9778a376f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:58.682421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c8442-799f-4560-ba4c-c2e9778a376f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:58.682728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c8442-799f-4560-ba4c-c2e9778a376f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:58.683002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c8442-799f-4560-ba4c-c2e9778a376f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:58.683718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c8442-799f-4560-ba4c-c2e9778a376f None admin] RBAC: Adding query filter params (interface=public, service_id=3da4e868ddd2499d93c27ca2a305ae15, region_id=RegionOne) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:58.684026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c8442-799f-4560-ba4c-c2e9778a376f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:58.685545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c8442-799f-4560-ba4c-c2e9778a376f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:58.698261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 389/772] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Sep 9 16:18:58 2022] GET /identity/v3/endpoints?service_id=3da4e868ddd2499d93c27ca2a305ae15&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:58.704486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658910e2-db11-4ce0-9adb-15751fd496f1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:58.760105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bba03e-6ae0-431a-878f-bbee0e5c4ff4 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:58.760301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bba03e-6ae0-431a-878f-bbee0e5c4ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:58.760560 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bba03e-6ae0-431a-878f-bbee0e5c4ff4 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:58.762663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 390/773] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:18:58.768151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c0860-0b4e-4ccf-a80f-c4537e82e383 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:58.768430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c0860-0b4e-4ccf-a80f-c4537e82e383 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:58.768706 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c0860-0b4e-4ccf-a80f-c4537e82e383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:58.789335 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-111c0860-0b4e-4ccf-a80f-c4537e82e383 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:58.790602 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-111c0860-0b4e-4ccf-a80f-c4537e82e383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:58.835281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658910e2-db11-4ce0-9adb-15751fd496f1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:58.905463 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88371cbb-c117-4d14-9d5d-17612910a44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:58.907389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88371cbb-c117-4d14-9d5d-17612910a44e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:58.907671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88371cbb-c117-4d14-9d5d-17612910a44e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:58.907942 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88371cbb-c117-4d14-9d5d-17612910a44e None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:58.908559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88371cbb-c117-4d14-9d5d-17612910a44e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:58.909940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88371cbb-c117-4d14-9d5d-17612910a44e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:58.914222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 391/774] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:18:58.918167 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 384/775] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Sep 9 16:18:58 2022] GET /identity/v3/services => generated 772 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:59.276876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c0860-0b4e-4ccf-a80f-c4537e82e383 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:59.352422 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:59.387292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c0860-0b4e-4ccf-a80f-c4537e82e383 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:59.422950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbff444-88eb-48b2-97ea-a9900b2a2a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:59.423631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbff444-88eb-48b2-97ea-a9900b2a2a67 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:59.423896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbff444-88eb-48b2-97ea-a9900b2a2a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:59.424163 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbff444-88eb-48b2-97ea-a9900b2a2a67 None admin] PATH_INFO: `/v3/services/network` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:59.424780 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbff444-88eb-48b2-97ea-a9900b2a2a67 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:59.426341 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbff444-88eb-48b2-97ea-a9900b2a2a67 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-ffbff444-88eb-48b2-97ea-a9900b2a2a67 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:18:59.436739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:18:59.437793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:18:59.438639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 392/776] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:18:59 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:18:59.445278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88371cbb-c117-4d14-9d5d-17612910a44e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:59.524301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:59.552196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815be444-be7c-4759-ab77-33f2db3423c0 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:59.552696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815be444-be7c-4759-ab77-33f2db3423c0 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:59.553065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815be444-be7c-4759-ab77-33f2db3423c0 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:59.554822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 393/777] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:18:59.562569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef219e9-6cb7-4a7b-b9f4-dccd755995c1 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:59.563712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef219e9-6cb7-4a7b-b9f4-dccd755995c1 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:59.564300 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef219e9-6cb7-4a7b-b9f4-dccd755995c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:59.601731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-cef219e9-6cb7-4a7b-b9f4-dccd755995c1 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:18:59.603842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cef219e9-6cb7-4a7b-b9f4-dccd755995c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:18:59.607913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88371cbb-c117-4d14-9d5d-17612910a44e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:59.680148 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73884d3-6aee-42e5-b9da-63049259c88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:59.680776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73884d3-6aee-42e5-b9da-63049259c88c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:59.681053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73884d3-6aee-42e5-b9da-63049259c88c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:59.681320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73884d3-6aee-42e5-b9da-63049259c88c None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:59.681950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73884d3-6aee-42e5-b9da-63049259c88c None admin] RBAC: Adding query filter params (name=network) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:59.682304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73884d3-6aee-42e5-b9da-63049259c88c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:59.683749 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73884d3-6aee-42e5-b9da-63049259c88c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:59.688068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 394/778] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:18:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:18:59.697757 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 385/779] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Sep 9 16:18:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:59.703566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef219e9-6cb7-4a7b-b9f4-dccd755995c1 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:18:59.797803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:18:59.850387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef219e9-6cb7-4a7b-b9f4-dccd755995c1 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:18:59.896310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ecb02-08aa-4455-a711-e0da1d3bda9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:18:59.896853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ecb02-08aa-4455-a711-e0da1d3bda9c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:18:59.897134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ecb02-08aa-4455-a711-e0da1d3bda9c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:18:59.897408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ecb02-08aa-4455-a711-e0da1d3bda9c None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:18:59.898058 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ecb02-08aa-4455-a711-e0da1d3bda9c None admin] RBAC: Adding query filter params (type=network) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:18:59.898350 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ecb02-08aa-4455-a711-e0da1d3bda9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:18:59.899763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ecb02-08aa-4455-a711-e0da1d3bda9c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:18:59.908482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 395/780] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Sep 9 16:18:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:18:59.914319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73884d3-6aee-42e5-b9da-63049259c88c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:00.002355 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ecb02-08aa-4455-a711-e0da1d3bda9c None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:00.096462 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73884d3-6aee-42e5-b9da-63049259c88c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:00.121811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:00.149270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ecb02-08aa-4455-a711-e0da1d3bda9c None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:00.178373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:00.187553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c25db-9d98-4f3f-9554-19a75f12bcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:00.188848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c25db-9d98-4f3f-9554-19a75f12bcda None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:00.189794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c25db-9d98-4f3f-9554-19a75f12bcda None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:00.191157 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c25db-9d98-4f3f-9554-19a75f12bcda None admin] PATH_INFO: `/v3/endpoints` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:00.192099 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c25db-9d98-4f3f-9554-19a75f12bcda None admin] RBAC: Adding query filter params (interface=public, service_id=3c2fe6841cfa47868be2f3558b8304a9, region_id=RegionOne) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:00.193024 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c25db-9d98-4f3f-9554-19a75f12bcda None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:00.195019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c25db-9d98-4f3f-9554-19a75f12bcda None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:00.198151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83968300-65ec-401b-940c-27ef6d28fce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:00.199063 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83968300-65ec-401b-940c-27ef6d28fce4 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:00.199555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83968300-65ec-401b-940c-27ef6d28fce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:00.200036 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83968300-65ec-401b-940c-27ef6d28fce4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:00.213924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83968300-65ec-401b-940c-27ef6d28fce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:19:00.214865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 386/781] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Sep 9 16:18:59 2022] GET /identity/v3/endpoints?service_id=3c2fe6841cfa47868be2f3558b8304a9&interface=public®ion_id=RegionOne => generated 194 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-83968300-65ec-401b-940c-27ef6d28fce4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:19:00.219766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:00.220694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:00.221543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:00.221543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:00.221543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:00.221543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:00.221543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:19:00.221543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:19:00.221543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:19:00.221543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:19:00.221543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:19:00.221543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:19:00.221543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:19:00.221543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:00.222177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c25db-9d98-4f3f-9554-19a75f12bcda None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:00.229201 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 396/782] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:19:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:00.231630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83968300-65ec-401b-940c-27ef6d28fce4 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:00.353747 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83968300-65ec-401b-940c-27ef6d28fce4 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:00.371076 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c25db-9d98-4f3f-9554-19a75f12bcda None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:00.396170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f50cd6-359c-4d5e-bfea-14c0758cce60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:00.396823 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f50cd6-359c-4d5e-bfea-14c0758cce60 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:00.397100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f50cd6-359c-4d5e-bfea-14c0758cce60 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:00.397369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f50cd6-359c-4d5e-bfea-14c0758cce60 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:00.398464 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f50cd6-359c-4d5e-bfea-14c0758cce60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:00.398820 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f50cd6-359c-4d5e-bfea-14c0758cce60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:00.400501 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f50cd6-359c-4d5e-bfea-14c0758cce60 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:00.409656 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42824fea-f5fd-46ca-98bf-1e0bd728bf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:00.410579 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42824fea-f5fd-46ca-98bf-1e0bd728bf0a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:00.414526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42824fea-f5fd-46ca-98bf-1e0bd728bf0a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:00.414646 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 397/783] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:19:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:00.414646 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937e6bd-6801-4247-b158-f2bddcffa2a9 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:00.414646 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937e6bd-6801-4247-b158-f2bddcffa2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:00.414646 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937e6bd-6801-4247-b158-f2bddcffa2a9 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:00.416737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42824fea-f5fd-46ca-98bf-1e0bd728bf0a None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:00.417634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42824fea-f5fd-46ca-98bf-1e0bd728bf0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:00.418245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 398/784] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:00.419353 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e0f11-d1ec-401c-9946-d0651341cc92 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:00.419630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e0f11-d1ec-401c-9946-d0651341cc92 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:00.419899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e0f11-d1ec-401c-9946-d0651341cc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:00.420702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42824fea-f5fd-46ca-98bf-1e0bd728bf0a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:00.434699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 387/785] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Sep 9 16:19:00 2022] GET /identity/v3/services => generated 772 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:00.436112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42824fea-f5fd-46ca-98bf-1e0bd728bf0a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:00.440109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-df0e0f11-d1ec-401c-9946-d0651341cc92 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:00.442719 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-df0e0f11-d1ec-401c-9946-d0651341cc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:00.498090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:00.509571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:00.530760 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 399/786] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:00.621925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42824fea-f5fd-46ca-98bf-1e0bd728bf0a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:00.671492 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b45008-475a-400c-b58d-2b5d8d6fd35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:00.672084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b45008-475a-400c-b58d-2b5d8d6fd35f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:00.672363 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b45008-475a-400c-b58d-2b5d8d6fd35f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:00.672634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b45008-475a-400c-b58d-2b5d8d6fd35f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:00.685807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b45008-475a-400c-b58d-2b5d8d6fd35f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:00.687347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b45008-475a-400c-b58d-2b5d8d6fd35f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-90b45008-475a-400c-b58d-2b5d8d6fd35f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:00.711247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:00.712266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:00.713142 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:00.713142 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:00.713142 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:00.713142 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 388/787] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:00.718418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e0f11-d1ec-401c-9946-d0651341cc92 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:00.833938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e0f11-d1ec-401c-9946-d0651341cc92 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:00.850952 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b45008-475a-400c-b58d-2b5d8d6fd35f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:00.874779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08aa83-0a95-4bfa-817c-cf5e562490c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:00.876192 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08aa83-0a95-4bfa-817c-cf5e562490c5 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:00.876957 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08aa83-0a95-4bfa-817c-cf5e562490c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:00.877425 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08aa83-0a95-4bfa-817c-cf5e562490c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:00.878428 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08aa83-0a95-4bfa-817c-cf5e562490c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:00.879269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08aa83-0a95-4bfa-817c-cf5e562490c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:00.881414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08aa83-0a95-4bfa-817c-cf5e562490c5 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:00.896016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 400/788] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:00.902678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08aa83-0a95-4bfa-817c-cf5e562490c5 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:00.966994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:01.012238 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08aa83-0a95-4bfa-817c-cf5e562490c5 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:01.021931 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:01.040378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b45008-475a-400c-b58d-2b5d8d6fd35f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:01.060880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3527bac-130f-4200-b487-9e1c3d37176d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:01.061891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3527bac-130f-4200-b487-9e1c3d37176d None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:01.062364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3527bac-130f-4200-b487-9e1c3d37176d None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:01.062847 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3527bac-130f-4200-b487-9e1c3d37176d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:01.081573 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3527bac-130f-4200-b487-9e1c3d37176d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:01.083444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3527bac-130f-4200-b487-9e1c3d37176d None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-a3527bac-130f-4200-b487-9e1c3d37176d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:19:01.103926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:01.104922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:01.105824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:01.105824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:01.105824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:19:01.105824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:19:01.105824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:19:01.105824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:19:01.105824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:19:01.105824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:19:01.105824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:19:01.105824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:01.106336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 401/789] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Sep 9 16:19:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:01.112049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3527bac-130f-4200-b487-9e1c3d37176d None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:01.119637 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11718e45-3037-4a56-b689-209e56a3772b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:01.120287 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11718e45-3037-4a56-b689-209e56a3772b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:01.120563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11718e45-3037-4a56-b689-209e56a3772b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:01.120833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11718e45-3037-4a56-b689-209e56a3772b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:01.141655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11718e45-3037-4a56-b689-209e56a3772b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-11718e45-3037-4a56-b689-209e56a3772b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:19:01.148616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:01.149602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:01.150405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:01.150405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:01.150405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:01.150405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:01.150405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:19:01.150405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:19:01.150405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:19:01.150405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:19:01.150405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:19:01.150405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:19:01.150405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:19:01.150405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:01.150405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 389/790] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:19:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:01.151772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533614a-a935-4f70-bb70-8aa1709e55f2 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:01.152380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533614a-a935-4f70-bb70-8aa1709e55f2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:01.152653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533614a-a935-4f70-bb70-8aa1709e55f2 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:01.154566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 390/791] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:01.156105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533614a-a935-4f70-bb70-8aa1709e55f2 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:01.231998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:01.239369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3527bac-130f-4200-b487-9e1c3d37176d None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:01.280534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb27bab-14b8-4dd8-9204-d40f07058eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:01.281339 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb27bab-14b8-4dd8-9204-d40f07058eff None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:01.281689 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb27bab-14b8-4dd8-9204-d40f07058eff None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:01.282018 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb27bab-14b8-4dd8-9204-d40f07058eff None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:01.282868 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb27bab-14b8-4dd8-9204-d40f07058eff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:01.283235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb27bab-14b8-4dd8-9204-d40f07058eff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:01.283426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533614a-a935-4f70-bb70-8aa1709e55f2 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:01.285060 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb27bab-14b8-4dd8-9204-d40f07058eff None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:01.310453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 402/792] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Sep 9 16:19:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:01.311829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6fdb2-0e33-467c-a033-29f045feedd4 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:01.312179 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6fdb2-0e33-467c-a033-29f045feedd4 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:01.312517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6fdb2-0e33-467c-a033-29f045feedd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:01.323695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af33ea-d069-44da-b864-fa8603ba7fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:01.324591 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af33ea-d069-44da-b864-fa8603ba7fce None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:01.325083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af33ea-d069-44da-b864-fa8603ba7fce None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:01.325581 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af33ea-d069-44da-b864-fa8603ba7fce None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:01.326984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af33ea-d069-44da-b864-fa8603ba7fce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:01.327473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af33ea-d069-44da-b864-fa8603ba7fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:01.329541 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af33ea-d069-44da-b864-fa8603ba7fce None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:01.333763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-63e6fdb2-0e33-467c-a033-29f045feedd4 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:01.335098 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-63e6fdb2-0e33-467c-a033-29f045feedd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:01.340507 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 391/793] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:19:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:01.341583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af33ea-d069-44da-b864-fa8603ba7fce None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:01.364647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:01.430901 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 403/794] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:01.432529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6fdb2-0e33-467c-a033-29f045feedd4 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:01.506370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af33ea-d069-44da-b864-fa8603ba7fce None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:01.547073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e938be-7867-47b1-b783-5c004e39f28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:01.550902 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e938be-7867-47b1-b783-5c004e39f28d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:01.551890 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e938be-7867-47b1-b783-5c004e39f28d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:01.552424 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e938be-7867-47b1-b783-5c004e39f28d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:01.561899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:01.584704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6fdb2-0e33-467c-a033-29f045feedd4 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:01.627231 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12227487-bb29-44c3-aa26-12aab9cd260f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:01.627781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12227487-bb29-44c3-aa26-12aab9cd260f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:01.628057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12227487-bb29-44c3-aa26-12aab9cd260f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:01.628326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12227487-bb29-44c3-aa26-12aab9cd260f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:01.638537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12227487-bb29-44c3-aa26-12aab9cd260f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:01.639314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e938be-7867-47b1-b783-5c004e39f28d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:01.639990 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12227487-bb29-44c3-aa26-12aab9cd260f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:01.641405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e938be-7867-47b1-b783-5c004e39f28d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-12227487-bb29-44c3-aa26-12aab9cd260f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:01.654751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:01.655731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:01.656633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:01.656633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:01.656633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:01.656633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 404/795] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:01.657424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8b4af-e3c4-4320-a91b-7d9b8d70e9f7 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:01.657699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8b4af-e3c4-4320-a91b-7d9b8d70e9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:01.657979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8b4af-e3c4-4320-a91b-7d9b8d70e9f7 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:01.661933 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:01.662766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 405/796] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:01.664338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8b4af-e3c4-4320-a91b-7d9b8d70e9f7 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:01.759087 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 392/797] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:19:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:19:01.760669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3d44a-b2bf-471f-848f-6969e8ca6f97 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:01.761151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3d44a-b2bf-471f-848f-6969e8ca6f97 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:01.761629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3d44a-b2bf-471f-848f-6969e8ca6f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:01.772864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8b4af-e3c4-4320-a91b-7d9b8d70e9f7 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:01.779258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-48a3d44a-b2bf-471f-848f-6969e8ca6f97 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:01.780432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-48a3d44a-b2bf-471f-848f-6969e8ca6f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:01.807759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39284747-83e7-4d80-85ad-8b2011d86747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:01.808364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39284747-83e7-4d80-85ad-8b2011d86747 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:01.808743 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39284747-83e7-4d80-85ad-8b2011d86747 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:01.809064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39284747-83e7-4d80-85ad-8b2011d86747 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:01.809860 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39284747-83e7-4d80-85ad-8b2011d86747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:01.810163 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39284747-83e7-4d80-85ad-8b2011d86747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:01.811604 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39284747-83e7-4d80-85ad-8b2011d86747 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:01.813828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:01.826441 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 406/798] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:01.827301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39284747-83e7-4d80-85ad-8b2011d86747 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:01.858045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 393/799] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:01.859111 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3d44a-b2bf-471f-848f-6969e8ca6f97 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:01.903289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:01.965989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3d44a-b2bf-471f-848f-6969e8ca6f97 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:01.966396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39284747-83e7-4d80-85ad-8b2011d86747 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:02.005136 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e12d7-0f10-44e0-b422-73236502d01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:02.005858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e12d7-0f10-44e0-b422-73236502d01c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:02.006133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e12d7-0f10-44e0-b422-73236502d01c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:02.006400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e12d7-0f10-44e0-b422-73236502d01c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:02.011791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78233758-20b0-40da-b439-7f0cd3cc6919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:02.012660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78233758-20b0-40da-b439-7f0cd3cc6919 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:02.013640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78233758-20b0-40da-b439-7f0cd3cc6919 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:02.014108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78233758-20b0-40da-b439-7f0cd3cc6919 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:02.016663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e12d7-0f10-44e0-b422-73236502d01c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:02.018108 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e12d7-0f10-44e0-b422-73236502d01c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:02.028088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78233758-20b0-40da-b439-7f0cd3cc6919 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-354e12d7-0f10-44e0-b422-73236502d01c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:02.032881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:02.034132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:02.035021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:02.035021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:02.035021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:02.035021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 394/800] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:02.035021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e12d7-0f10-44e0-b422-73236502d01c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-78233758-20b0-40da-b439-7f0cd3cc6919 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:19:02.036242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:02.037211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:02.038694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:02.038694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:02.038694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:02.038694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:02.038694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:19:02.038694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:19:02.038694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:19:02.038694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:19:02.038694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:19:02.038694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:19:02.038694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:19:02.038694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:02.042110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 407/801] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:19:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:02.043082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78233758-20b0-40da-b439-7f0cd3cc6919 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:02.120657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:02.150803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:02.157762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e12d7-0f10-44e0-b422-73236502d01c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:02.166698 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78233758-20b0-40da-b439-7f0cd3cc6919 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:02.194935 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7f746-c362-4403-a293-40159fe000dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:02.195674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7f746-c362-4403-a293-40159fe000dd None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:02.195999 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7f746-c362-4403-a293-40159fe000dd None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:02.196277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7f746-c362-4403-a293-40159fe000dd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:02.199896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2f5bd-f5fe-4155-8e51-94cd6af452b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:02.200503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2f5bd-f5fe-4155-8e51-94cd6af452b7 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:02.200773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2f5bd-f5fe-4155-8e51-94cd6af452b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:02.201036 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2f5bd-f5fe-4155-8e51-94cd6af452b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:02.201665 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2f5bd-f5fe-4155-8e51-94cd6af452b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:02.201961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2f5bd-f5fe-4155-8e51-94cd6af452b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:02.203360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2f5bd-f5fe-4155-8e51-94cd6af452b7 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:02.204663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7f746-c362-4403-a293-40159fe000dd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:02.206192 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7f746-c362-4403-a293-40159fe000dd None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:02.215252 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 408/802] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:02.216327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2f5bd-f5fe-4155-8e51-94cd6af452b7 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-d2b7f746-c362-4403-a293-40159fe000dd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:19:02.218030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:02.218982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:02.219953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:02.219953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:02.219953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:19:02.219953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:19:02.219953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:19:02.219953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:19:02.219953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:19:02.219953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:19:02.219953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 16:19:02.219953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:02.219953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 395/803] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:19:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:02.220531 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7f746-c362-4403-a293-40159fe000dd None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:02.321624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7f746-c362-4403-a293-40159fe000dd None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:02.325656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2f5bd-f5fe-4155-8e51-94cd6af452b7 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:02.341809 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:02.357316 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03946889-728f-4059-9cef-50ff8a54c809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:02.357940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03946889-728f-4059-9cef-50ff8a54c809 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:02.358215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03946889-728f-4059-9cef-50ff8a54c809 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:02.358514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03946889-728f-4059-9cef-50ff8a54c809 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:02.359072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03946889-728f-4059-9cef-50ff8a54c809 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:02.359612 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab503b60-707b-4c38-a375-accb2b5f5ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:02.360220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab503b60-707b-4c38-a375-accb2b5f5ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:02.360432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03946889-728f-4059-9cef-50ff8a54c809 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:02.360529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab503b60-707b-4c38-a375-accb2b5f5ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:02.360791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab503b60-707b-4c38-a375-accb2b5f5ce9 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:02.361406 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab503b60-707b-4c38-a375-accb2b5f5ce9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:02.361708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab503b60-707b-4c38-a375-accb2b5f5ce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:02.363127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab503b60-707b-4c38-a375-accb2b5f5ce9 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-03946889-728f-4059-9cef-50ff8a54c809 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:02.369427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:02.370400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:02.371961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 396/804] 10.222.0.23 () {56 vars in 1129 bytes} [Fri Sep 9 16:19:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:02.372782 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03946889-728f-4059-9cef-50ff8a54c809 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:02.382388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 409/805] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:19:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:02.382388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab503b60-707b-4c38-a375-accb2b5f5ce9 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:02.397840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:02.477177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03946889-728f-4059-9cef-50ff8a54c809 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:02.488585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab503b60-707b-4c38-a375-accb2b5f5ce9 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:02.513779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7c624-1bbe-4c86-8c8f-085b6b5a9f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:02.514439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7c624-1bbe-4c86-8c8f-085b6b5a9f2f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:02.514760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7c624-1bbe-4c86-8c8f-085b6b5a9f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:02.515025 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7c624-1bbe-4c86-8c8f-085b6b5a9f2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:02.521724 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7c624-1bbe-4c86-8c8f-085b6b5a9f2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:02.523209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7c624-1bbe-4c86-8c8f-085b6b5a9f2f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:02.523406 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24acba-4611-4282-808c-de7af2df2c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:02.524017 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24acba-4611-4282-808c-de7af2df2c34 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:02.524294 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24acba-4611-4282-808c-de7af2df2c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:02.524559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24acba-4611-4282-808c-de7af2df2c34 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:02.525431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24acba-4611-4282-808c-de7af2df2c34 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:02.525863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24acba-4611-4282-808c-de7af2df2c34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:02.527288 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24acba-4611-4282-808c-de7af2df2c34 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c7f7c624-1bbe-4c86-8c8f-085b6b5a9f2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:19:02.534400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:02.535312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:02.541410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:19:02.541410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:19:02.541410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:19:02.541410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:19:02.541410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:19:02.541410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:19:02.541410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:02.541410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:02.541410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 397/806] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:02.541410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7c624-1bbe-4c86-8c8f-085b6b5a9f2f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:02.553280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 410/807] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Sep 9 16:19:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:02.554251 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24acba-4611-4282-808c-de7af2df2c34 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:02.592649 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:02.628054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:02.649590 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7c624-1bbe-4c86-8c8f-085b6b5a9f2f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:02.657862 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24acba-4611-4282-808c-de7af2df2c34 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:02.685618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582465e1-4c3b-4bfc-9343-da925378ff50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:02.686229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582465e1-4c3b-4bfc-9343-da925378ff50 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:02.686534 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582465e1-4c3b-4bfc-9343-da925378ff50 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:02.686804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582465e1-4c3b-4bfc-9343-da925378ff50 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:02.687437 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582465e1-4c3b-4bfc-9343-da925378ff50 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:02.687748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582465e1-4c3b-4bfc-9343-da925378ff50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:02.689116 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582465e1-4c3b-4bfc-9343-da925378ff50 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:02.692710 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284319b8-4c74-4205-a4a0-7f718271e764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:02.693682 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284319b8-4c74-4205-a4a0-7f718271e764 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:02.693993 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 398/808] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Sep 9 16:19:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:02.694273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284319b8-4c74-4205-a4a0-7f718271e764 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:02.694699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284319b8-4c74-4205-a4a0-7f718271e764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:02.694795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582465e1-4c3b-4bfc-9343-da925378ff50 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:02.708406 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284319b8-4c74-4205-a4a0-7f718271e764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:02.709930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284319b8-4c74-4205-a4a0-7f718271e764 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-284319b8-4c74-4205-a4a0-7f718271e764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:02.725039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:02.726149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:02.727351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:02.727351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:02.727351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:02.730984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 411/809] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:02.731814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284319b8-4c74-4205-a4a0-7f718271e764 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:02.803904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582465e1-4c3b-4bfc-9343-da925378ff50 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:02.830855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:02.838393 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1270b1-60b4-4b72-8b13-6d140d00d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:02.839039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1270b1-60b4-4b72-8b13-6d140d00d99c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:02.839319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1270b1-60b4-4b72-8b13-6d140d00d99c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:02.839618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1270b1-60b4-4b72-8b13-6d140d00d99c None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:02.840306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1270b1-60b4-4b72-8b13-6d140d00d99c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:02.840617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1270b1-60b4-4b72-8b13-6d140d00d99c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:02.842050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1270b1-60b4-4b72-8b13-6d140d00d99c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:02.844753 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284319b8-4c74-4205-a4a0-7f718271e764 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:02.859564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 399/810] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:19:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:02.859564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1270b1-60b4-4b72-8b13-6d140d00d99c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:02.862367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:02.890579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eef1b4-fc8a-45cf-b38b-1c5c5652cca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:02.891272 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eef1b4-fc8a-45cf-b38b-1c5c5652cca3 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:02.891553 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eef1b4-fc8a-45cf-b38b-1c5c5652cca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:02.891819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eef1b4-fc8a-45cf-b38b-1c5c5652cca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:02.903778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eef1b4-fc8a-45cf-b38b-1c5c5652cca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:02.905265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eef1b4-fc8a-45cf-b38b-1c5c5652cca3 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-29eef1b4-fc8a-45cf-b38b-1c5c5652cca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:02.919970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:02.920928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:02.921802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:02.921802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:02.921802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:02.921802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 412/811] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:02.922035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eef1b4-fc8a-45cf-b38b-1c5c5652cca3 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:02.968001 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1270b1-60b4-4b72-8b13-6d140d00d99c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:03.006433 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a951ec-f63d-43c8-9fa0-22ad0d0abe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:03.007080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a951ec-f63d-43c8-9fa0-22ad0d0abe5d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:03.007356 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a951ec-f63d-43c8-9fa0-22ad0d0abe5d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:03.007651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a951ec-f63d-43c8-9fa0-22ad0d0abe5d None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:03.008275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a951ec-f63d-43c8-9fa0-22ad0d0abe5d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:03.008574 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a951ec-f63d-43c8-9fa0-22ad0d0abe5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:03.009938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a951ec-f63d-43c8-9fa0-22ad0d0abe5d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:03.014681 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 400/812] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Sep 9 16:19:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:03.015478 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a951ec-f63d-43c8-9fa0-22ad0d0abe5d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:03.027086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eef1b4-fc8a-45cf-b38b-1c5c5652cca3 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:03.059106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ceb4cf-5d80-4b7f-b3a7-9463e8ebda22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:03.059707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceb4cf-5d80-4b7f-b3a7-9463e8ebda22 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:03.059983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceb4cf-5d80-4b7f-b3a7-9463e8ebda22 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:03.060232 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceb4cf-5d80-4b7f-b3a7-9463e8ebda22 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:03.060872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ceb4cf-5d80-4b7f-b3a7-9463e8ebda22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:03.061165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ceb4cf-5d80-4b7f-b3a7-9463e8ebda22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:03.062553 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ceb4cf-5d80-4b7f-b3a7-9463e8ebda22 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:03.071336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:03.076513 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 413/813] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:03.077580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ceb4cf-5d80-4b7f-b3a7-9463e8ebda22 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:03.085457 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:03.117820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a951ec-f63d-43c8-9fa0-22ad0d0abe5d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:03.150880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973e787-da17-4056-af59-b5ad504b6fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:03.151484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973e787-da17-4056-af59-b5ad504b6fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:03.151742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973e787-da17-4056-af59-b5ad504b6fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:03.152008 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973e787-da17-4056-af59-b5ad504b6fa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:03.152728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973e787-da17-4056-af59-b5ad504b6fa8 None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=152a41ad598b4b01b1881d928c53c848) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:03.153034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973e787-da17-4056-af59-b5ad504b6fa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:03.154515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973e787-da17-4056-af59-b5ad504b6fa8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:03.160130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 401/814] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:19:03 2022] GET /identity/v3/role_assignments?user.id=152a41ad598b4b01b1881d928c53c848&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=c6da7ba2be504943af29338ee8855f05 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:03.161248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800deae6-457f-46d1-ba48-82403c72eeec None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:03.161526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800deae6-457f-46d1-ba48-82403c72eeec None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:03.161828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800deae6-457f-46d1-ba48-82403c72eeec None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:03.163188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 402/815] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:03.163708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800deae6-457f-46d1-ba48-82403c72eeec None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:03.185726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ceb4cf-5d80-4b7f-b3a7-9463e8ebda22 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:03.225332 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f99915-26eb-431b-9ad5-e8312fb59d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:03.225972 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f99915-26eb-431b-9ad5-e8312fb59d91 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:03.226254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f99915-26eb-431b-9ad5-e8312fb59d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:03.226557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f99915-26eb-431b-9ad5-e8312fb59d91 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:03.227233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f99915-26eb-431b-9ad5-e8312fb59d91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:03.228167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f99915-26eb-431b-9ad5-e8312fb59d91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:03.229597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f99915-26eb-431b-9ad5-e8312fb59d91 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:03.242426 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 414/816] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:03.243258 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f99915-26eb-431b-9ad5-e8312fb59d91 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:03.286220 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800deae6-457f-46d1-ba48-82403c72eeec None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:03.310016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:03.319060 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:03.324511 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b117004-53ab-4888-a6ac-57165b0f125f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:03.325211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b117004-53ab-4888-a6ac-57165b0f125f None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:03.325548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b117004-53ab-4888-a6ac-57165b0f125f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:03.325877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b117004-53ab-4888-a6ac-57165b0f125f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:03.326645 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b117004-53ab-4888-a6ac-57165b0f125f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:03.328133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b117004-53ab-4888-a6ac-57165b0f125f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:03.355341 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 403/817] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:19:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:03.357078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3beaa-db3d-4369-a83f-a80e5e3859a7 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:03.357426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3beaa-db3d-4369-a83f-a80e5e3859a7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:03.357798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3beaa-db3d-4369-a83f-a80e5e3859a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:03.360691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f99915-26eb-431b-9ad5-e8312fb59d91 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:03.381847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-dfa3beaa-db3d-4369-a83f-a80e5e3859a7 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:03.383039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dfa3beaa-db3d-4369-a83f-a80e5e3859a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:03.418030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b573fe5-100f-4f0f-bc4f-925bba379f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:03.418927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b573fe5-100f-4f0f-bc4f-925bba379f63 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:03.419201 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b573fe5-100f-4f0f-bc4f-925bba379f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:03.419484 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b573fe5-100f-4f0f-bc4f-925bba379f63 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:03.431159 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b573fe5-100f-4f0f-bc4f-925bba379f63 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:03.432642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b573fe5-100f-4f0f-bc4f-925bba379f63 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-3b573fe5-100f-4f0f-bc4f-925bba379f63 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:19:03.451233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:03.452244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:03.453150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:03.453150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:03.453150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:19:03.453150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:19:03.453150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:19:03.453150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:19:03.453150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:19:03.453150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:19:03.453150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 16:19:03.453150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:03.453150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 415/818] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Sep 9 16:19:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:03.453719 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b573fe5-100f-4f0f-bc4f-925bba379f63 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:03.477404 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 404/819] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:03.478360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3beaa-db3d-4369-a83f-a80e5e3859a7 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:03.575508 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b573fe5-100f-4f0f-bc4f-925bba379f63 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:03.592635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:03.612218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a0577-b9f2-4725-bc3c-804509ff6354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:03.612858 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a0577-b9f2-4725-bc3c-804509ff6354 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:03.613133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a0577-b9f2-4725-bc3c-804509ff6354 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:03.613398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a0577-b9f2-4725-bc3c-804509ff6354 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:03.621678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a0577-b9f2-4725-bc3c-804509ff6354 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:03.623204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a0577-b9f2-4725-bc3c-804509ff6354 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:03.631843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3beaa-db3d-4369-a83f-a80e5e3859a7 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-ca6a0577-b9f2-4725-bc3c-804509ff6354 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:19:03.638025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:03.638922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:03.639849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:19:03.639849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:19:03.639849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:19:03.639849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:19:03.639849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:19:03.639849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:19:03.639849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:03.639849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:03.639849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 416/820] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:03.646833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a0577-b9f2-4725-bc3c-804509ff6354 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:03.652388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:03.685642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2a826-26a9-4cad-95bb-8533a93d2a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:03.686255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2a826-26a9-4cad-95bb-8533a93d2a74 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:03.686552 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2a826-26a9-4cad-95bb-8533a93d2a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:03.686822 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2a826-26a9-4cad-95bb-8533a93d2a74 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:03.687622 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2a826-26a9-4cad-95bb-8533a93d2a74 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:03.687923 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2a826-26a9-4cad-95bb-8533a93d2a74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:03.689372 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2a826-26a9-4cad-95bb-8533a93d2a74 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:03.708135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 405/821] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Sep 9 16:19:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:03.714411 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2a826-26a9-4cad-95bb-8533a93d2a74 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:03.779735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a0577-b9f2-4725-bc3c-804509ff6354 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:03.819460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfd632-3cc3-4099-9f53-4413ccd3638d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:03.820654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfd632-3cc3-4099-9f53-4413ccd3638d None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:03.821027 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfd632-3cc3-4099-9f53-4413ccd3638d None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:03.821391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfd632-3cc3-4099-9f53-4413ccd3638d None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:03.822552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecfd632-3cc3-4099-9f53-4413ccd3638d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:03.822954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecfd632-3cc3-4099-9f53-4413ccd3638d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:03.825080 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecfd632-3cc3-4099-9f53-4413ccd3638d None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:03.839468 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 417/822] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:19:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:03.840790 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfd632-3cc3-4099-9f53-4413ccd3638d None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:03.846696 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2a826-26a9-4cad-95bb-8533a93d2a74 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:03.882095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:03.886447 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76dd20-eb06-4c9e-a7ff-cd6bb354407d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:03.887194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76dd20-eb06-4c9e-a7ff-cd6bb354407d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:03.887951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76dd20-eb06-4c9e-a7ff-cd6bb354407d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:03.888271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76dd20-eb06-4c9e-a7ff-cd6bb354407d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:03.903302 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76dd20-eb06-4c9e-a7ff-cd6bb354407d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:03.905192 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76dd20-eb06-4c9e-a7ff-cd6bb354407d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-3f76dd20-eb06-4c9e-a7ff-cd6bb354407d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:03.925188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:03.926331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:03.928146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:03.928146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:03.928146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:03.928146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 406/823] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:03.928146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76dd20-eb06-4c9e-a7ff-cd6bb354407d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:03.962707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfd632-3cc3-4099-9f53-4413ccd3638d None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:03.963455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:04.007474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3e98c-49f9-4105-951c-d5b60c33c185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:04.008396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3e98c-49f9-4105-951c-d5b60c33c185 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:04.008813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3e98c-49f9-4105-951c-d5b60c33c185 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:04.009230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3e98c-49f9-4105-951c-d5b60c33c185 None admin] PATH_INFO: `/v3/services/network` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:04.010020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3e98c-49f9-4105-951c-d5b60c33c185 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:04.011605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3e98c-49f9-4105-951c-d5b60c33c185 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-c9e3e98c-49f9-4105-951c-d5b60c33c185 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:04.022453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:04.023696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:04.024716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:04.024716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:04.024716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:04.024716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:04.024716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:04.024716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:04.024716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:04.024716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:04.024716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:04.024716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:04.024716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:04.024716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:04.024716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 16:19:04.024716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:04.027061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 418/824] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:03 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:04.028037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3e98c-49f9-4105-951c-d5b60c33c185 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:04.046097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76dd20-eb06-4c9e-a7ff-cd6bb354407d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:04.098922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed87e2e-3b6f-44db-b1d6-274d6ef223da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:04.099716 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed87e2e-3b6f-44db-b1d6-274d6ef223da None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:04.100006 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed87e2e-3b6f-44db-b1d6-274d6ef223da None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:04.100320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed87e2e-3b6f-44db-b1d6-274d6ef223da None admin] PATH_INFO: `/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/e926034945b44087b378b3468954a18a/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:04.102262 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed87e2e-3b6f-44db-b1d6-274d6ef223da None admin] RBAC: Authorizing `identity:create_grant(project_id=253ff30a56444df28d44c4b7eaadb276, user_id=e926034945b44087b378b3468954a18a, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:04.103758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed87e2e-3b6f-44db-b1d6-274d6ef223da None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:04.120133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 407/825] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:19:03 2022] PUT /identity/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/e926034945b44087b378b3468954a18a/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:19:04.120444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed87e2e-3b6f-44db-b1d6-274d6ef223da None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:04.158114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:04.163573 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3e98c-49f9-4105-951c-d5b60c33c185 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:04.216533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afb263-6f75-4937-a78d-09fba78008b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:04.217267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afb263-6f75-4937-a78d-09fba78008b2 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:04.217548 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afb263-6f75-4937-a78d-09fba78008b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:04.217824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afb263-6f75-4937-a78d-09fba78008b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:04.218510 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afb263-6f75-4937-a78d-09fba78008b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:04.218825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afb263-6f75-4937-a78d-09fba78008b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:04.220268 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afb263-6f75-4937-a78d-09fba78008b2 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:04.247064 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:04.257522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 419/826] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:04.269999 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed87e2e-3b6f-44db-b1d6-274d6ef223da None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:04.276054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afb263-6f75-4937-a78d-09fba78008b2 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:04.358392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e09b13-e7bc-439f-a887-b7a4eba7ca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:04.359054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e09b13-e7bc-439f-a887-b7a4eba7ca52 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:04.359327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e09b13-e7bc-439f-a887-b7a4eba7ca52 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:04.359590 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e09b13-e7bc-439f-a887-b7a4eba7ca52 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:04.360224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e09b13-e7bc-439f-a887-b7a4eba7ca52 None admin] RBAC: Adding query filter params (name=network) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:04.360519 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e09b13-e7bc-439f-a887-b7a4eba7ca52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:04.361938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e09b13-e7bc-439f-a887-b7a4eba7ca52 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:04.375167 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 408/827] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Sep 9 16:19:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:04.382490 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e09b13-e7bc-439f-a887-b7a4eba7ca52 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:04.472070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afb263-6f75-4937-a78d-09fba78008b2 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:04.538003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989348ac-32d6-40ad-bf05-944ff82fec8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:04.538668 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989348ac-32d6-40ad-bf05-944ff82fec8a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:04.538944 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989348ac-32d6-40ad-bf05-944ff82fec8a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:04.539209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989348ac-32d6-40ad-bf05-944ff82fec8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:04.552127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989348ac-32d6-40ad-bf05-944ff82fec8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:04.553693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989348ac-32d6-40ad-bf05-944ff82fec8a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:04.559729 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e09b13-e7bc-439f-a887-b7a4eba7ca52 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-989348ac-32d6-40ad-bf05-944ff82fec8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:19:04.573139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:04.574079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:04.575023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:19:04.575023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:19:04.575023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:19:04.575023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:19:04.575023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:19:04.575023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:19:04.575023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:04.575023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:04.575023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 420/828] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:04.580132 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989348ac-32d6-40ad-bf05-944ff82fec8a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:04.618166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:04.627215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8182da-d590-4392-9b4e-6303c6dc7395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:04.627835 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8182da-d590-4392-9b4e-6303c6dc7395 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:04.628119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8182da-d590-4392-9b4e-6303c6dc7395 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:04.628379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8182da-d590-4392-9b4e-6303c6dc7395 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:04.629007 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8182da-d590-4392-9b4e-6303c6dc7395 None admin] RBAC: Adding query filter params (type=network) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:04.629305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8182da-d590-4392-9b4e-6303c6dc7395 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:04.630743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8182da-d590-4392-9b4e-6303c6dc7395 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:04.638745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 409/829] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Sep 9 16:19:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:04.651497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8182da-d590-4392-9b4e-6303c6dc7395 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:04.688869 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:04.840422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8182da-d590-4392-9b4e-6303c6dc7395 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:04.877239 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f91cf-8408-47ca-80a4-ec010d1e0b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:04.878080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f91cf-8408-47ca-80a4-ec010d1e0b73 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:04.878526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f91cf-8408-47ca-80a4-ec010d1e0b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:04.879101 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f91cf-8408-47ca-80a4-ec010d1e0b73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:04.880326 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f91cf-8408-47ca-80a4-ec010d1e0b73 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:04.882093 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f91cf-8408-47ca-80a4-ec010d1e0b73 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:04.896418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989348ac-32d6-40ad-bf05-944ff82fec8a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:04.906964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 410/830] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:19:04 2022] POST /identity/v3/endpoints => generated 337 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:04.963433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137115db-8636-483c-b95a-9036f909df18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:04.964260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137115db-8636-483c-b95a-9036f909df18 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:04.964634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137115db-8636-483c-b95a-9036f909df18 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:04.965002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137115db-8636-483c-b95a-9036f909df18 None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:04.965807 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137115db-8636-483c-b95a-9036f909df18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:04.966207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137115db-8636-483c-b95a-9036f909df18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:04.967760 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137115db-8636-483c-b95a-9036f909df18 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:04.986462 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 421/831] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:19:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:04.992733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f91cf-8408-47ca-80a4-ec010d1e0b73 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:05.101728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:05.142666 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f91cf-8408-47ca-80a4-ec010d1e0b73 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:05.180332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bac496-9e6e-4c4a-a97b-f79232d1118b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:05.181299 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bac496-9e6e-4c4a-a97b-f79232d1118b None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:05.181712 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bac496-9e6e-4c4a-a97b-f79232d1118b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:05.182143 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bac496-9e6e-4c4a-a97b-f79232d1118b None admin] PATH_INFO: `/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/048b4abace244e48b6897daec0277e2f/roles/2b68b618a1b846f69bbe9aa08852a66f` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:05.197870 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bac496-9e6e-4c4a-a97b-f79232d1118b None admin] RBAC: Authorizing `identity:create_grant(project_id=253ff30a56444df28d44c4b7eaadb276, user_id=048b4abace244e48b6897daec0277e2f, role_id=2b68b618a1b846f69bbe9aa08852a66f)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:05.199532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bac496-9e6e-4c4a-a97b-f79232d1118b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:05.219220 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 411/832] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:19:04 2022] PUT /identity/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/048b4abace244e48b6897daec0277e2f/roles/2b68b618a1b846f69bbe9aa08852a66f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:19:05.779595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f3f5d-20a8-414c-9d2d-fcd81bbc092b None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:05.780162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f3f5d-20a8-414c-9d2d-fcd81bbc092b None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:05.780162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f3f5d-20a8-414c-9d2d-fcd81bbc092b None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:05.781877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 422/833] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:05.787355 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fd747-3825-4cc6-a57e-d92fdb726376 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:05.787634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fd747-3825-4cc6-a57e-d92fdb726376 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:05.787898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fd747-3825-4cc6-a57e-d92fdb726376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:05.806623 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-dd2fd747-3825-4cc6-a57e-d92fdb726376 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:05.807822 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dd2fd747-3825-4cc6-a57e-d92fdb726376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:05.881730 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 412/834] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:05.945205 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619fb6d-0a87-450e-b4c0-bc185fb5f778 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:05.945478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619fb6d-0a87-450e-b4c0-bc185fb5f778 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:05.945889 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619fb6d-0a87-450e-b4c0-bc185fb5f778 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:05.947577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 423/835] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:05.952837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74c12a-ac87-4c2b-89e5-7cba443b3d83 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:05.953198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74c12a-ac87-4c2b-89e5-7cba443b3d83 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:05.953540 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74c12a-ac87-4c2b-89e5-7cba443b3d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:05.971981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1d74c12a-ac87-4c2b-89e5-7cba443b3d83 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:05.973156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1d74c12a-ac87-4c2b-89e5-7cba443b3d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:06.025182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:06.051099 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 413/836] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:06.158615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619fb6d-0a87-450e-b4c0-bc185fb5f778 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:06.239438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:06.267218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619fb6d-0a87-450e-b4c0-bc185fb5f778 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:06.301206 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c5e41-0b0a-4e62-bf88-30872c030270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:06.302100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c5e41-0b0a-4e62-bf88-30872c030270 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:06.302602 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c5e41-0b0a-4e62-bf88-30872c030270 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:06.303063 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c5e41-0b0a-4e62-bf88-30872c030270 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:06.303806 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c5e41-0b0a-4e62-bf88-30872c030270 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:06.305445 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c5e41-0b0a-4e62-bf88-30872c030270 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-c47c5e41-0b0a-4e62-bf88-30872c030270 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:06.314391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:06.315822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:06.316688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:06.316688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:06.316688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:06.316688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:06.316688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:06.316688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:06.316688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:06.316688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:06.316688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:06.316688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:06.316688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:06.316688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:06.316688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 16:19:06.316688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:06.319090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 424/837] 10.222.0.23 () {56 vars in 1132 bytes} [Fri Sep 9 16:19:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:06.326669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74c12a-ac87-4c2b-89e5-7cba443b3d83 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:06.328700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c5e41-0b0a-4e62-bf88-30872c030270 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:06.442823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74c12a-ac87-4c2b-89e5-7cba443b3d83 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:06.451182 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c5e41-0b0a-4e62-bf88-30872c030270 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:06.481797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2e779-8652-4873-b76e-3159a24478e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:06.482639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2e779-8652-4873-b76e-3159a24478e2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:06.482920 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2e779-8652-4873-b76e-3159a24478e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:06.483188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2e779-8652-4873-b76e-3159a24478e2 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:06.483869 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2e779-8652-4873-b76e-3159a24478e2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:06.484176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2e779-8652-4873-b76e-3159a24478e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:06.485549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2e779-8652-4873-b76e-3159a24478e2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:06.490120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 414/838] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Sep 9 16:19:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:06.492285 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2311805-f6f3-4c0c-9003-9e6afb7b66fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:06.492909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2311805-f6f3-4c0c-9003-9e6afb7b66fb None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:06.493179 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2311805-f6f3-4c0c-9003-9e6afb7b66fb None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:06.493439 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2311805-f6f3-4c0c-9003-9e6afb7b66fb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:06.494010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2311805-f6f3-4c0c-9003-9e6afb7b66fb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:06.495453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2311805-f6f3-4c0c-9003-9e6afb7b66fb None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:06.495665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db3464-ade5-4277-9c5b-ff8bff47bbf3 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:06.495947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db3464-ade5-4277-9c5b-ff8bff47bbf3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:06.496217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db3464-ade5-4277-9c5b-ff8bff47bbf3 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:06.498808 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 415/839] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:06.498808 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db3464-ade5-4277-9c5b-ff8bff47bbf3 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-f2311805-f6f3-4c0c-9003-9e6afb7b66fb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:06.505136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:06.506170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:06.511479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 425/840] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:19:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:06.512270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc851a-9856-4708-808c-91d2b5e3aa12 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:06.512270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc851a-9856-4708-808c-91d2b5e3aa12 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:06.512270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc851a-9856-4708-808c-91d2b5e3aa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:06.517042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:06.528397 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-8acc851a-9856-4708-808c-91d2b5e3aa12 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:06.529588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8acc851a-9856-4708-808c-91d2b5e3aa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:06.552056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:06.600846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 426/841] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:06.602064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc851a-9856-4708-808c-91d2b5e3aa12 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:06.614023 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db3464-ade5-4277-9c5b-ff8bff47bbf3 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:06.650581 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb52434-456b-4176-a0bc-e5305cc237ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:06.651192 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb52434-456b-4176-a0bc-e5305cc237ab None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:06.651464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb52434-456b-4176-a0bc-e5305cc237ab None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:06.651727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb52434-456b-4176-a0bc-e5305cc237ab None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:06.652349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb52434-456b-4176-a0bc-e5305cc237ab None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:06.652643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb52434-456b-4176-a0bc-e5305cc237ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:06.654001 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb52434-456b-4176-a0bc-e5305cc237ab None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:06.659126 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 416/842] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Sep 9 16:19:06 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:06.725127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc851a-9856-4708-808c-91d2b5e3aa12 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:06.763905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a70ca8-9ac5-4db9-ae69-77d0f9ad45d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:06.764932 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a70ca8-9ac5-4db9-ae69-77d0f9ad45d3 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:06.765406 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a70ca8-9ac5-4db9-ae69-77d0f9ad45d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:06.765880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a70ca8-9ac5-4db9-ae69-77d0f9ad45d3 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:06.766767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a70ca8-9ac5-4db9-ae69-77d0f9ad45d3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:06.767287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a70ca8-9ac5-4db9-ae69-77d0f9ad45d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:06.769282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a70ca8-9ac5-4db9-ae69-77d0f9ad45d3 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:06.771353 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:06.777735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 427/843] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:19:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:06.784118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb52434-456b-4176-a0bc-e5305cc237ab None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:06.893212 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb52434-456b-4176-a0bc-e5305cc237ab None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:06.903953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:06.914854 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a70ca8-9ac5-4db9-ae69-77d0f9ad45d3 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:06.928320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eed637-b56b-497b-a7c1-f418f5c6a200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:06.928929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eed637-b56b-497b-a7c1-f418f5c6a200 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:06.929205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eed637-b56b-497b-a7c1-f418f5c6a200 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:06.929475 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eed637-b56b-497b-a7c1-f418f5c6a200 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:06.930165 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eed637-b56b-497b-a7c1-f418f5c6a200 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:06.930464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eed637-b56b-497b-a7c1-f418f5c6a200 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:06.931853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eed637-b56b-497b-a7c1-f418f5c6a200 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:06.936758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 417/844] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:19:06 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:07.031495 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a70ca8-9ac5-4db9-ae69-77d0f9ad45d3 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:07.072485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31d05a-ebde-46b6-82a7-f8048e4447ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:07.073517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31d05a-ebde-46b6-82a7-f8048e4447ef None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:07.074013 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31d05a-ebde-46b6-82a7-f8048e4447ef None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:07.074474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31d05a-ebde-46b6-82a7-f8048e4447ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:07.087211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c31d05a-ebde-46b6-82a7-f8048e4447ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-8c31d05a-ebde-46b6-82a7-f8048e4447ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:19:07.092779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:07.093640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:07.094864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:07.094864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:07.094864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:07.094864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:07.094864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:19:07.094864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:19:07.094864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:19:07.094864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:19:07.094864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:19:07.094864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:19:07.094864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:19:07.094864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:07.094864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 428/845] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:19:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:07.101115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eed637-b56b-497b-a7c1-f418f5c6a200 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:07.207516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eed637-b56b-497b-a7c1-f418f5c6a200 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:07.241758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85335bef-bbe8-4f43-a42d-73f2109cb4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:07.242369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85335bef-bbe8-4f43-a42d-73f2109cb4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:07.242677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85335bef-bbe8-4f43-a42d-73f2109cb4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:07.242948 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85335bef-bbe8-4f43-a42d-73f2109cb4f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:07.243682 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85335bef-bbe8-4f43-a42d-73f2109cb4f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:07.243980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85335bef-bbe8-4f43-a42d-73f2109cb4f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:07.245401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85335bef-bbe8-4f43-a42d-73f2109cb4f0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:07.254114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 418/846] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:19:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:07.260099 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31d05a-ebde-46b6-82a7-f8048e4447ef None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:07.322071 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:07.365209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31d05a-ebde-46b6-82a7-f8048e4447ef None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:07.405095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccf4cb-f9c5-4a75-91f6-c557e8838aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:07.405810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccf4cb-f9c5-4a75-91f6-c557e8838aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:07.406087 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccf4cb-f9c5-4a75-91f6-c557e8838aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:07.406356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccf4cb-f9c5-4a75-91f6-c557e8838aa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:07.418640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccf4cb-f9c5-4a75-91f6-c557e8838aa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:07.420107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccf4cb-f9c5-4a75-91f6-c557e8838aa1 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-83ccf4cb-f9c5-4a75-91f6-c557e8838aa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:07.441640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:07.443750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:07.445543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:07.445543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:07.445543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:07.445543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 429/847] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:07.448857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85335bef-bbe8-4f43-a42d-73f2109cb4f0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:07.489387 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:07.508143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b845b1-1567-492b-8676-49c12af8fe82 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:07.508456 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b845b1-1567-492b-8676-49c12af8fe82 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:07.508682 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b845b1-1567-492b-8676-49c12af8fe82 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:07.510175 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 430/848] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:19:07.517801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db7a64-1b86-437b-a8be-a54727735d90 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:07.517801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db7a64-1b86-437b-a8be-a54727735d90 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:07.517801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db7a64-1b86-437b-a8be-a54727735d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:07.540361 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-79db7a64-1b86-437b-a8be-a54727735d90 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:07.541566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-79db7a64-1b86-437b-a8be-a54727735d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:07.566444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85335bef-bbe8-4f43-a42d-73f2109cb4f0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:07.608989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc17f03-6812-4aeb-8d4a-6b1192061964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:07.609966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc17f03-6812-4aeb-8d4a-6b1192061964 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:07.610247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc17f03-6812-4aeb-8d4a-6b1192061964 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:07.610542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc17f03-6812-4aeb-8d4a-6b1192061964 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:07.611379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc17f03-6812-4aeb-8d4a-6b1192061964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:07.611683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc17f03-6812-4aeb-8d4a-6b1192061964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:07.613741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc17f03-6812-4aeb-8d4a-6b1192061964 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:07.620279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 431/849] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:19:07.627306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 419/850] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:07.632570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db7a64-1b86-437b-a8be-a54727735d90 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:07.674903 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:07.734847 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db7a64-1b86-437b-a8be-a54727735d90 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:07.769008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c051a2-0f27-4671-894c-3bf1638a3f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:07.769569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c051a2-0f27-4671-894c-3bf1638a3f96 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:07.769846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c051a2-0f27-4671-894c-3bf1638a3f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:07.770170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c051a2-0f27-4671-894c-3bf1638a3f96 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:07.777412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c051a2-0f27-4671-894c-3bf1638a3f96 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:07.778884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c051a2-0f27-4671-894c-3bf1638a3f96 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-d7c051a2-0f27-4671-894c-3bf1638a3f96 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:19:07.790718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:07.791640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:07.792588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:07.792588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:07.792588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:19:07.792588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:19:07.792588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:19:07.792588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:19:07.792588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:19:07.792588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:19:07.792588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 16:19:07.792588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:07.792588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 432/851] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:19:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:07.799629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc17f03-6812-4aeb-8d4a-6b1192061964 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:07.883476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c051a2-0f27-4671-894c-3bf1638a3f96 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:07.899751 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:07.906112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc17f03-6812-4aeb-8d4a-6b1192061964 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:07.944261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64d589-ba7e-4a53-8cc3-92549893568b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:07.944971 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64d589-ba7e-4a53-8cc3-92549893568b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:07.945251 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64d589-ba7e-4a53-8cc3-92549893568b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:07.945607 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64d589-ba7e-4a53-8cc3-92549893568b None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:07.946336 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64d589-ba7e-4a53-8cc3-92549893568b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:07.946662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64d589-ba7e-4a53-8cc3-92549893568b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:07.948103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64d589-ba7e-4a53-8cc3-92549893568b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:07.965981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 420/852] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Sep 9 16:19:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:07.975508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64d589-ba7e-4a53-8cc3-92549893568b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:08.002517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:08.006455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c051a2-0f27-4671-894c-3bf1638a3f96 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:08.046842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6174f9-6a17-4ad3-9970-22f625d96c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:08.047449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6174f9-6a17-4ad3-9970-22f625d96c97 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:08.047731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6174f9-6a17-4ad3-9970-22f625d96c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:08.047998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6174f9-6a17-4ad3-9970-22f625d96c97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:08.059011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6174f9-6a17-4ad3-9970-22f625d96c97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-ab6174f9-6a17-4ad3-9970-22f625d96c97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:19:08.066837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:08.067834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:08.069038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:08.069038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:08.069038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:08.069038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:08.069038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:19:08.069038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:19:08.069038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:19:08.069038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:19:08.069038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:19:08.069038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:19:08.069038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:19:08.069038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:08.069038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 433/853] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:19:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:08.077148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6174f9-6a17-4ad3-9970-22f625d96c97 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:08.081817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64d589-ba7e-4a53-8cc3-92549893568b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:08.118795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859dcc0-accf-487e-8bad-a55632e9826f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:08.119487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859dcc0-accf-487e-8bad-a55632e9826f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:08.119827 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859dcc0-accf-487e-8bad-a55632e9826f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:08.120163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859dcc0-accf-487e-8bad-a55632e9826f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:08.144558 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:08.189731 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859dcc0-accf-487e-8bad-a55632e9826f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:08.191759 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859dcc0-accf-487e-8bad-a55632e9826f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:08.193073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6174f9-6a17-4ad3-9970-22f625d96c97 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:08.237329 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb6810-a134-46f3-8fb7-255ea1261397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:08.238208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb6810-a134-46f3-8fb7-255ea1261397 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:08.238488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb6810-a134-46f3-8fb7-255ea1261397 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:08.238788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb6810-a134-46f3-8fb7-255ea1261397 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:08.239458 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb6810-a134-46f3-8fb7-255ea1261397 None admin] RBAC: Adding query filter params (name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:08.239782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb6810-a134-46f3-8fb7-255ea1261397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:08.241176 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb6810-a134-46f3-8fb7-255ea1261397 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:08.250157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 434/854] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:19:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:08.257407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb6810-a134-46f3-8fb7-255ea1261397 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:08.286527 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:08.297398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 421/855] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:19:07 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:19:08.303151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859dcc0-accf-487e-8bad-a55632e9826f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:08.360791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:08.372537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb6810-a134-46f3-8fb7-255ea1261397 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:08.407817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb1320-4612-4232-ace5-6ea08a75210b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:08.408441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859dcc0-accf-487e-8bad-a55632e9826f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:08.409007 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb1320-4612-4232-ace5-6ea08a75210b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:08.409462 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb1320-4612-4232-ace5-6ea08a75210b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:08.409987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb1320-4612-4232-ace5-6ea08a75210b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:08.421400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb1320-4612-4232-ace5-6ea08a75210b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:08.423313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb1320-4612-4232-ace5-6ea08a75210b None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-d2eb1320-4612-4232-ace5-6ea08a75210b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:08.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:08.439492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:08.440728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:08.440728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:08.440728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:08.442908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 435/856] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:08.447927 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01050816-76e2-4511-ac0d-6f4a91f6d1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:08.448564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050816-76e2-4511-ac0d-6f4a91f6d1af None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:08.449439 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb1320-4612-4232-ace5-6ea08a75210b None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:08.450166 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050816-76e2-4511-ac0d-6f4a91f6d1af None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:08.450643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050816-76e2-4511-ac0d-6f4a91f6d1af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:08.466327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01050816-76e2-4511-ac0d-6f4a91f6d1af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:08.468388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01050816-76e2-4511-ac0d-6f4a91f6d1af None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-01050816-76e2-4511-ac0d-6f4a91f6d1af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:08.489003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:08.490049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:08.491597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:08.491597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:08.491597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:08.495111 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 422/857] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:08.500340 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01050816-76e2-4511-ac0d-6f4a91f6d1af None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:08.560751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:08.591590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb1320-4612-4232-ace5-6ea08a75210b None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:08.624223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01050816-76e2-4511-ac0d-6f4a91f6d1af None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:08.631950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:08.633360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de09361-cbbf-4a7b-a0af-ef2a8ddcc9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:08.634199 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de09361-cbbf-4a7b-a0af-ef2a8ddcc9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:08.634835 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de09361-cbbf-4a7b-a0af-ef2a8ddcc9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:08.635402 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de09361-cbbf-4a7b-a0af-ef2a8ddcc9c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:08.636201 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de09361-cbbf-4a7b-a0af-ef2a8ddcc9c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:08.638867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de09361-cbbf-4a7b-a0af-ef2a8ddcc9c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:08.638867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de09361-cbbf-4a7b-a0af-ef2a8ddcc9c0 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:08.653099 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 436/858] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:08.660451 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de09361-cbbf-4a7b-a0af-ef2a8ddcc9c0 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:08.664184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513e958-b51e-4c50-b9c5-10baae81a62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:08.664802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e958-b51e-4c50-b9c5-10baae81a62f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:08.665078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e958-b51e-4c50-b9c5-10baae81a62f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:08.666089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e958-b51e-4c50-b9c5-10baae81a62f None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:08.666089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513e958-b51e-4c50-b9c5-10baae81a62f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:08.666259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513e958-b51e-4c50-b9c5-10baae81a62f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:08.667675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513e958-b51e-4c50-b9c5-10baae81a62f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:08.684256 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 423/859] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:08.685306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513e958-b51e-4c50-b9c5-10baae81a62f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:08.774879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de09361-cbbf-4a7b-a0af-ef2a8ddcc9c0 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:08.817074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513e958-b51e-4c50-b9c5-10baae81a62f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:08.819243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c9d4d-325d-42f8-a158-c8a84179e97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:08.820319 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c9d4d-325d-42f8-a158-c8a84179e97e None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:08.820695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c9d4d-325d-42f8-a158-c8a84179e97e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:08.821080 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c9d4d-325d-42f8-a158-c8a84179e97e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:08.824687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:08.832455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c9d4d-325d-42f8-a158-c8a84179e97e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:08.834670 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c9d4d-325d-42f8-a158-c8a84179e97e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-796c9d4d-325d-42f8-a158-c8a84179e97e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:19:08.849544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:08.850784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:08.851739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:08.851739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:08.851739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:19:08.851739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:19:08.851739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:19:08.851739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:19:08.851739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:19:08.851739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:19:08.851739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 16:19:08.851739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:08.854641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 437/860] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Sep 9 16:19:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:08.858609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c9d4d-325d-42f8-a158-c8a84179e97e None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:08.881924 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcebfc9-1489-48f9-a0f3-4e281986aa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:08.882863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcebfc9-1489-48f9-a0f3-4e281986aa9f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:08.883232 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcebfc9-1489-48f9-a0f3-4e281986aa9f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:08.883614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcebfc9-1489-48f9-a0f3-4e281986aa9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:08.891512 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcebfc9-1489-48f9-a0f3-4e281986aa9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:08.893268 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcebfc9-1489-48f9-a0f3-4e281986aa9f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-2dcebfc9-1489-48f9-a0f3-4e281986aa9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:19:08.906141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:08.907288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:08.908369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:19:08.908369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:19:08.908369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:19:08.908369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:19:08.908369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:19:08.908369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:19:08.908369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:08.908369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:08.913867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 424/861] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:08.914992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcebfc9-1489-48f9-a0f3-4e281986aa9f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:08.945338 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:08.970008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c9d4d-325d-42f8-a158-c8a84179e97e None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:09.003134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c10b54-3431-4976-a6bd-215e05c64c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:09.003757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c10b54-3431-4976-a6bd-215e05c64c4e None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:09.004007 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c10b54-3431-4976-a6bd-215e05c64c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:09.004276 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c10b54-3431-4976-a6bd-215e05c64c4e None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:09.005003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c10b54-3431-4976-a6bd-215e05c64c4e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:09.005307 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c10b54-3431-4976-a6bd-215e05c64c4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:09.006785 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c10b54-3431-4976-a6bd-215e05c64c4e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:09.023910 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 438/862] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Sep 9 16:19:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:09.024946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe50fe-3532-48f8-9d1a-444e4013cd30 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:09.025223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe50fe-3532-48f8-9d1a-444e4013cd30 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:09.025488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe50fe-3532-48f8-9d1a-444e4013cd30 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:09.026825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 439/863] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:09.031327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcebfc9-1489-48f9-a0f3-4e281986aa9f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:09.031473 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe50fe-3532-48f8-9d1a-444e4013cd30 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:09.067554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a24ee-cc74-4528-895c-4922d40b53aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:09.068034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a24ee-cc74-4528-895c-4922d40b53aa None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:09.068313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a24ee-cc74-4528-895c-4922d40b53aa None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:09.068581 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a24ee-cc74-4528-895c-4922d40b53aa None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:09.069484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a24ee-cc74-4528-895c-4922d40b53aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:09.069593 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a24ee-cc74-4528-895c-4922d40b53aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:09.071038 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a24ee-cc74-4528-895c-4922d40b53aa None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:09.094271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 425/864] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:19:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:09.094271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30169832-7d1d-4841-a076-9ad5fef4c18b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:09.094271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30169832-7d1d-4841-a076-9ad5fef4c18b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:09.094271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30169832-7d1d-4841-a076-9ad5fef4c18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:09.103214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:09.111717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-30169832-7d1d-4841-a076-9ad5fef4c18b None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:09.112678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-30169832-7d1d-4841-a076-9ad5fef4c18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:09.155412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe50fe-3532-48f8-9d1a-444e4013cd30 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:09.185061 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 426/865] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:09.188072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30169832-7d1d-4841-a076-9ad5fef4c18b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:09.193896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6945a7-acc7-45e9-98ce-f1ffd296a92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:09.194746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6945a7-acc7-45e9-98ce-f1ffd296a92c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:09.195028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6945a7-acc7-45e9-98ce-f1ffd296a92c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:09.195307 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6945a7-acc7-45e9-98ce-f1ffd296a92c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:09.210370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:09.277409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6945a7-acc7-45e9-98ce-f1ffd296a92c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:09.278892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6945a7-acc7-45e9-98ce-f1ffd296a92c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:09.305069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30169832-7d1d-4841-a076-9ad5fef4c18b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:09.330896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:09.340032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d513f7-ddd4-4bb0-951f-cb9483b78778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:09.340641 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d513f7-ddd4-4bb0-951f-cb9483b78778 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:09.340915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d513f7-ddd4-4bb0-951f-cb9483b78778 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:09.341180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d513f7-ddd4-4bb0-951f-cb9483b78778 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:09.341904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d513f7-ddd4-4bb0-951f-cb9483b78778 None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=e926034945b44087b378b3468954a18a) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:09.342207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d513f7-ddd4-4bb0-951f-cb9483b78778 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:09.343670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d513f7-ddd4-4bb0-951f-cb9483b78778 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:09.350613 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 427/866] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:19:09 2022] GET /identity/v3/role_assignments?user.id=e926034945b44087b378b3468954a18a&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=c6da7ba2be504943af29338ee8855f05 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:09.350974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce85d9-d045-4d05-8b5a-0ba2ccf80d92 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:09.351255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce85d9-d045-4d05-8b5a-0ba2ccf80d92 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:09.351525 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce85d9-d045-4d05-8b5a-0ba2ccf80d92 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:09.357272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 428/867] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:09.360870 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd3fab-8b73-4ea5-adb3-460e940dea7a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:09.361450 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd3fab-8b73-4ea5-adb3-460e940dea7a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:09.361956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd3fab-8b73-4ea5-adb3-460e940dea7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:09.381870 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-36dd3fab-8b73-4ea5-adb3-460e940dea7a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:09.383504 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-36dd3fab-8b73-4ea5-adb3-460e940dea7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:09.391296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 440/868] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:19:09 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:19:09.399499 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6945a7-acc7-45e9-98ce-f1ffd296a92c None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:09.469733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 429/869] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:09.470642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd3fab-8b73-4ea5-adb3-460e940dea7a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:09.484029 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:09.528564 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6945a7-acc7-45e9-98ce-f1ffd296a92c None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:09.582098 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a35669-e006-4934-8668-748dba4d4cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:09.583343 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a35669-e006-4934-8668-748dba4d4cbb None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:09.583864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a35669-e006-4934-8668-748dba4d4cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:09.584292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a35669-e006-4934-8668-748dba4d4cbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:09.597866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a35669-e006-4934-8668-748dba4d4cbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:09.599500 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a35669-e006-4934-8668-748dba4d4cbb None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:09.605998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd3fab-8b73-4ea5-adb3-460e940dea7a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-c7a35669-e006-4934-8668-748dba4d4cbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:09.618973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:09.620141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:09.621200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:09.621200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:09.621200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:09.621200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 441/870] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:09.625612 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a35669-e006-4934-8668-748dba4d4cbb None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:09.629501 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:09.660907 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66cae9-551c-4f49-a196-ed026f5f9845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:09.661933 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66cae9-551c-4f49-a196-ed026f5f9845 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:09.662644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66cae9-551c-4f49-a196-ed026f5f9845 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:09.663057 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66cae9-551c-4f49-a196-ed026f5f9845 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:09.664367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66cae9-551c-4f49-a196-ed026f5f9845 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:09.665997 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66cae9-551c-4f49-a196-ed026f5f9845 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:09.687842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 430/871] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Sep 9 16:19:09 2022] POST /identity/v3/services => generated 269 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:09.738255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66cae9-551c-4f49-a196-ed026f5f9845 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:09.746627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a35669-e006-4934-8668-748dba4d4cbb None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:09.796940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130106a-72c4-4707-94ed-ff75eaf24395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:09.797573 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130106a-72c4-4707-94ed-ff75eaf24395 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:09.797850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130106a-72c4-4707-94ed-ff75eaf24395 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:09.798118 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130106a-72c4-4707-94ed-ff75eaf24395 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:09.799024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130106a-72c4-4707-94ed-ff75eaf24395 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:09.799334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130106a-72c4-4707-94ed-ff75eaf24395 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:09.800867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130106a-72c4-4707-94ed-ff75eaf24395 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:09.815842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 442/872] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:09.821517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130106a-72c4-4707-94ed-ff75eaf24395 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:09.840787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:09.883119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66cae9-551c-4f49-a196-ed026f5f9845 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:09.898443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:09.927617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8a898-dc33-4359-9ab8-a399fb0b8fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:09.928952 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8a898-dc33-4359-9ab8-a399fb0b8fa9 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:09.928952 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8a898-dc33-4359-9ab8-a399fb0b8fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:09.928952 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8a898-dc33-4359-9ab8-a399fb0b8fa9 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:09.929413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8a898-dc33-4359-9ab8-a399fb0b8fa9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:09.933431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8a898-dc33-4359-9ab8-a399fb0b8fa9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:09.949675 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130106a-72c4-4707-94ed-ff75eaf24395 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:09.959481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 431/873] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Sep 9 16:19:09 2022] POST /identity/v3/services => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:19:10.001328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01b8db-456f-4e06-9f58-c8276675eed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:10.001989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01b8db-456f-4e06-9f58-c8276675eed9 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:10.002263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01b8db-456f-4e06-9f58-c8276675eed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:10.002563 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01b8db-456f-4e06-9f58-c8276675eed9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:10.010773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01b8db-456f-4e06-9f58-c8276675eed9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:10.012258 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01b8db-456f-4e06-9f58-c8276675eed9 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-fc01b8db-456f-4e06-9f58-c8276675eed9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:19:10.025755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:10.026757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:10.027633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:19:10.027633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:19:10.027633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:19:10.027633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:19:10.027633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:19:10.027633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:19:10.027633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:10.027633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:10.027633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 443/874] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:10.033144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8a898-dc33-4359-9ab8-a399fb0b8fa9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:10.153288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8a898-dc33-4359-9ab8-a399fb0b8fa9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:10.184052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:10.191775 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6cde4-6e0d-462f-82c1-b35b87e310a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:10.195399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6cde4-6e0d-462f-82c1-b35b87e310a3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:10.195399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6cde4-6e0d-462f-82c1-b35b87e310a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:10.195399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6cde4-6e0d-462f-82c1-b35b87e310a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:10.195399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6cde4-6e0d-462f-82c1-b35b87e310a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:10.195399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6cde4-6e0d-462f-82c1-b35b87e310a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:10.196541 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6cde4-6e0d-462f-82c1-b35b87e310a3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:10.210982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 432/875] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:19:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:10.219608 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01b8db-456f-4e06-9f58-c8276675eed9 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:10.319214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01b8db-456f-4e06-9f58-c8276675eed9 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:10.337746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:10.356356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b510e-9036-42f5-bf73-254cf63bbec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:10.356987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b510e-9036-42f5-bf73-254cf63bbec0 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:10.357262 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b510e-9036-42f5-bf73-254cf63bbec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:10.357530 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b510e-9036-42f5-bf73-254cf63bbec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:10.358279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b510e-9036-42f5-bf73-254cf63bbec0 None admin] RBAC: Adding query filter params (role.id=2b68b618a1b846f69bbe9aa08852a66f, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=048b4abace244e48b6897daec0277e2f) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:10.358619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b510e-9036-42f5-bf73-254cf63bbec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:10.360117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b510e-9036-42f5-bf73-254cf63bbec0 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:10.367311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 444/876] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:19:10 2022] GET /identity/v3/role_assignments?user.id=048b4abace244e48b6897daec0277e2f&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=2b68b618a1b846f69bbe9aa08852a66f => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:11.647735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c759d-f535-4908-b92d-bd8a3bb3c395 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:11.648267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c759d-f535-4908-b92d-bd8a3bb3c395 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:11.648267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c759d-f535-4908-b92d-bd8a3bb3c395 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:11.649820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 433/877] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:11.655363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0b48f-98a1-4af6-9014-dca4ae473542 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:11.655781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0b48f-98a1-4af6-9014-dca4ae473542 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:11.655969 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0b48f-98a1-4af6-9014-dca4ae473542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:11.674493 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-e0d0b48f-98a1-4af6-9014-dca4ae473542 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:11.675691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e0d0b48f-98a1-4af6-9014-dca4ae473542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:11.742989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 445/878] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:11.977236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411be38-8f5c-4740-8bae-759d1c7cf3b7 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:11.977506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411be38-8f5c-4740-8bae-759d1c7cf3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:11.977747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411be38-8f5c-4740-8bae-759d1c7cf3b7 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:11.979315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 434/879] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:11.984573 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac690a4-c667-4893-9fe1-d20ef6cbaa83 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:11.984858 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac690a4-c667-4893-9fe1-d20ef6cbaa83 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:11.985134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac690a4-c667-4893-9fe1-d20ef6cbaa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:12.003419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-1ac690a4-c667-4893-9fe1-d20ef6cbaa83 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:12.004601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1ac690a4-c667-4893-9fe1-d20ef6cbaa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:12.025632 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411be38-8f5c-4740-8bae-759d1c7cf3b7 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:12.085372 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 446/880] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:12.153934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411be38-8f5c-4740-8bae-759d1c7cf3b7 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:12.189050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ec1d1-8f26-47a5-a101-29df5eea2ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:12.189689 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec1d1-8f26-47a5-a101-29df5eea2ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:12.189966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec1d1-8f26-47a5-a101-29df5eea2ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:12.190620 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec1d1-8f26-47a5-a101-29df5eea2ed5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:12.191206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ec1d1-8f26-47a5-a101-29df5eea2ed5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:12.192582 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ec1d1-8f26-47a5-a101-29df5eea2ed5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-f88ec1d1-8f26-47a5-a101-29df5eea2ed5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:12.201876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:12.202980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:12.203876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 435/881] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Sep 9 16:19:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:12.211306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac690a4-c667-4893-9fe1-d20ef6cbaa83 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:12.215196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf97f1-5f4a-4371-b061-2bada1d65cfd None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:12.215485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf97f1-5f4a-4371-b061-2bada1d65cfd None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:12.215768 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf97f1-5f4a-4371-b061-2bada1d65cfd None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:12.217321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 436/882] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:19:12.222352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09d6a6-5479-4532-aef3-e6b615f5010c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:12.222667 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09d6a6-5479-4532-aef3-e6b615f5010c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:12.222938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09d6a6-5479-4532-aef3-e6b615f5010c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:12.241528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-6b09d6a6-5479-4532-aef3-e6b615f5010c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:12.242701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6b09d6a6-5479-4532-aef3-e6b615f5010c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:12.260282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:12.297891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:12.322723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 437/883] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:12 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:19:12.331746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac690a4-c667-4893-9fe1-d20ef6cbaa83 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:12.352010 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09d6a6-5479-4532-aef3-e6b615f5010c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:12.370040 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8a2e8-a1ae-4129-96f8-4eb8be61cc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:12.370722 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8a2e8-a1ae-4129-96f8-4eb8be61cc90 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:12.371005 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8a2e8-a1ae-4129-96f8-4eb8be61cc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:12.371277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8a2e8-a1ae-4129-96f8-4eb8be61cc90 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:12.372288 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8a2e8-a1ae-4129-96f8-4eb8be61cc90 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:12.372591 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8a2e8-a1ae-4129-96f8-4eb8be61cc90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:12.374167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8a2e8-a1ae-4129-96f8-4eb8be61cc90 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:12.379975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 447/884] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Sep 9 16:19:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:12.385704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8a2e8-a1ae-4129-96f8-4eb8be61cc90 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:12.488303 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09d6a6-5479-4532-aef3-e6b615f5010c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:12.495082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8a2e8-a1ae-4129-96f8-4eb8be61cc90 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:12.498931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:12.531295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf134dc6-f6eb-446d-9ec9-3e32b8fef72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:12.532098 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf134dc6-f6eb-446d-9ec9-3e32b8fef72a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:12.532380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf134dc6-f6eb-446d-9ec9-3e32b8fef72a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:12.532664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf134dc6-f6eb-446d-9ec9-3e32b8fef72a None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:12.533390 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf134dc6-f6eb-446d-9ec9-3e32b8fef72a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:12.533745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf134dc6-f6eb-446d-9ec9-3e32b8fef72a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:12.534416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b0661-e001-4785-8883-9bcd58ff3a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:12.535142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf134dc6-f6eb-446d-9ec9-3e32b8fef72a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:12.535515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b0661-e001-4785-8883-9bcd58ff3a7f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:12.535998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b0661-e001-4785-8883-9bcd58ff3a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:12.536860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b0661-e001-4785-8883-9bcd58ff3a7f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:12.538058 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b0661-e001-4785-8883-9bcd58ff3a7f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:12.539977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b0661-e001-4785-8883-9bcd58ff3a7f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:12.546907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 448/885] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Sep 9 16:19:12 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-274b0661-e001-4785-8883-9bcd58ff3a7f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:12.552327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:12.553311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:12.554205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:12.554205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:12.554205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:12.554205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:12.554205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:12.554205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:12.554205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:12.554205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:12.554205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:12.554205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:12.554205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:12.554205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:12.554205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 16:19:12.554205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:12.557336 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 438/886] 10.222.0.23 () {56 vars in 1132 bytes} [Fri Sep 9 16:19:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:12.563807 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf134dc6-f6eb-446d-9ec9-3e32b8fef72a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:12.571280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d947c-e9fd-46aa-82c7-5df4455cf5d1 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:12.571633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d947c-e9fd-46aa-82c7-5df4455cf5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:12.571830 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d947c-e9fd-46aa-82c7-5df4455cf5d1 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:12.573509 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 439/887] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:12.580286 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb44703-b0f9-4b27-86e8-aaa0f7a4b68f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:12.580612 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb44703-b0f9-4b27-86e8-aaa0f7a4b68f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:12.580973 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb44703-b0f9-4b27-86e8-aaa0f7a4b68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:12.600076 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0bb44703-b0f9-4b27-86e8-aaa0f7a4b68f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:12.601484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0bb44703-b0f9-4b27-86e8-aaa0f7a4b68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:12.651637 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:12.677604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 440/888] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:12 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:19:12.678821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb44703-b0f9-4b27-86e8-aaa0f7a4b68f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:12.692148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf134dc6-f6eb-446d-9ec9-3e32b8fef72a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:12.738347 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da239e34-6f80-48f3-8a4c-96f798dea872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:12.739044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da239e34-6f80-48f3-8a4c-96f798dea872 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:12.739335 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da239e34-6f80-48f3-8a4c-96f798dea872 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:12.739595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da239e34-6f80-48f3-8a4c-96f798dea872 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:12.740824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da239e34-6f80-48f3-8a4c-96f798dea872 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:12.740824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da239e34-6f80-48f3-8a4c-96f798dea872 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:12.741999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da239e34-6f80-48f3-8a4c-96f798dea872 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:12.750733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 449/889] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Sep 9 16:19:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:12.753121 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da239e34-6f80-48f3-8a4c-96f798dea872 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:12.788957 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:12.817266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb44703-b0f9-4b27-86e8-aaa0f7a4b68f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:12.861331 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da239e34-6f80-48f3-8a4c-96f798dea872 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:12.862510 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757966f-243a-45a7-917e-cda26c719690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:12.863444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757966f-243a-45a7-917e-cda26c719690 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:12.864563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757966f-243a-45a7-917e-cda26c719690 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:12.864971 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757966f-243a-45a7-917e-cda26c719690 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:12.865564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757966f-243a-45a7-917e-cda26c719690 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:12.866966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757966f-243a-45a7-917e-cda26c719690 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-f757966f-243a-45a7-917e-cda26c719690 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:12.878791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:12.879975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:12.880867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 441/890] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:19:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:12.885987 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757966f-243a-45a7-917e-cda26c719690 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:12.902214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08811eeb-7b87-4fee-8298-63d1df1db95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:12.902863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08811eeb-7b87-4fee-8298-63d1df1db95f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:12.903141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08811eeb-7b87-4fee-8298-63d1df1db95f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:12.903412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08811eeb-7b87-4fee-8298-63d1df1db95f None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:12.904046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08811eeb-7b87-4fee-8298-63d1df1db95f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:12.904346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08811eeb-7b87-4fee-8298-63d1df1db95f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:12.905705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08811eeb-7b87-4fee-8298-63d1df1db95f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:12.911127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 450/891] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Sep 9 16:19:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:12.919579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08811eeb-7b87-4fee-8298-63d1df1db95f None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:12.941119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:13.001103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757966f-243a-45a7-917e-cda26c719690 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:13.032393 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:13.033816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238359c7-1c12-4239-8e7b-7d92e8c91a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:13.034481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238359c7-1c12-4239-8e7b-7d92e8c91a86 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:13.034779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238359c7-1c12-4239-8e7b-7d92e8c91a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:13.035072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238359c7-1c12-4239-8e7b-7d92e8c91a86 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:13.035681 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238359c7-1c12-4239-8e7b-7d92e8c91a86 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:13.035982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238359c7-1c12-4239-8e7b-7d92e8c91a86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:13.037417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238359c7-1c12-4239-8e7b-7d92e8c91a86 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:13.040328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08811eeb-7b87-4fee-8298-63d1df1db95f None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:13.042120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 442/892] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:19:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:13.042935 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238359c7-1c12-4239-8e7b-7d92e8c91a86 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:13.080588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dc820-185a-489f-afb4-596c787709f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:13.081295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dc820-185a-489f-afb4-596c787709f5 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:13.081635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dc820-185a-489f-afb4-596c787709f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:13.081985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dc820-185a-489f-afb4-596c787709f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:13.082786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292dc820-185a-489f-afb4-596c787709f5 None admin] RBAC: Adding query filter params (interface=public, service_id=39cd728dbb9c40e6bc6bead7cf8bb13d, region_id=RegionOne) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:13.083155 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292dc820-185a-489f-afb4-596c787709f5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:13.084708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292dc820-185a-489f-afb4-596c787709f5 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:13.089822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 451/893] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Sep 9 16:19:12 2022] GET /identity/v3/endpoints?service_id=39cd728dbb9c40e6bc6bead7cf8bb13d&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:13.090634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dc820-185a-489f-afb4-596c787709f5 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:13.165315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238359c7-1c12-4239-8e7b-7d92e8c91a86 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:13.173346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:13.188715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dc820-185a-489f-afb4-596c787709f5 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:13.204043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d24096-5a98-4340-affa-2f3348e3b969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:13.204681 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d24096-5a98-4340-affa-2f3348e3b969 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:13.204958 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d24096-5a98-4340-affa-2f3348e3b969 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:13.205228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d24096-5a98-4340-affa-2f3348e3b969 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:13.215437 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d24096-5a98-4340-affa-2f3348e3b969 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-a3d24096-5a98-4340-affa-2f3348e3b969 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:19:13.220297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:13.221259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:13.222095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:13.222095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:13.222095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:13.222095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:13.222095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:19:13.222095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:19:13.222095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:19:13.222095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:19:13.222095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:19:13.222095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:19:13.222095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 16:19:13.222095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:13.222095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 443/894] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Sep 9 16:19:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:13.223483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d24096-5a98-4340-affa-2f3348e3b969 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:13.229440 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8fb85-bfcf-43f2-bdd6-b41210ef5fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:13.230068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8fb85-bfcf-43f2-bdd6-b41210ef5fdf None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:13.230346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8fb85-bfcf-43f2-bdd6-b41210ef5fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:13.230751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8fb85-bfcf-43f2-bdd6-b41210ef5fdf None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:13.231401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8fb85-bfcf-43f2-bdd6-b41210ef5fdf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:13.231704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8fb85-bfcf-43f2-bdd6-b41210ef5fdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:13.233496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8fb85-bfcf-43f2-bdd6-b41210ef5fdf None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:13.244780 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 452/895] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:19:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:13.244780 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8fb85-bfcf-43f2-bdd6-b41210ef5fdf None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:13.272721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:13.323974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d24096-5a98-4340-affa-2f3348e3b969 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:13.356361 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8fb85-bfcf-43f2-bdd6-b41210ef5fdf None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:13.359039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e51a6c-7c41-4001-b90e-8cbc7f979608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:13.359655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e51a6c-7c41-4001-b90e-8cbc7f979608 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:13.359930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e51a6c-7c41-4001-b90e-8cbc7f979608 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:13.360206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e51a6c-7c41-4001-b90e-8cbc7f979608 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:13.360819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e51a6c-7c41-4001-b90e-8cbc7f979608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:13.362196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e51a6c-7c41-4001-b90e-8cbc7f979608 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:13.369196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 444/896] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Sep 9 16:19:13 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:19:13.370564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e51a6c-7c41-4001-b90e-8cbc7f979608 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:13.397513 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d687c32-a368-4156-b551-6a1931d3cdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:13.398561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d687c32-a368-4156-b551-6a1931d3cdc3 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:13.399372 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d687c32-a368-4156-b551-6a1931d3cdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:13.399850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d687c32-a368-4156-b551-6a1931d3cdc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:13.401199 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d687c32-a368-4156-b551-6a1931d3cdc3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:13.401735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d687c32-a368-4156-b551-6a1931d3cdc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:13.404066 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d687c32-a368-4156-b551-6a1931d3cdc3 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:13.413249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:13.416731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 453/897] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Sep 9 16:19:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:13.423495 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d687c32-a368-4156-b551-6a1931d3cdc3 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:13.499894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e51a6c-7c41-4001-b90e-8cbc7f979608 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:13.526575 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:13.544143 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7b67b-70b5-47b3-b622-fefd9c032643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:13.544879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7b67b-70b5-47b3-b622-fefd9c032643 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:13.545229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7b67b-70b5-47b3-b622-fefd9c032643 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:13.545575 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7b67b-70b5-47b3-b622-fefd9c032643 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:13.546364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7b67b-70b5-47b3-b622-fefd9c032643 None admin] RBAC: Adding query filter params (interface=public, service_id=039b5bb67f954dd48328d423bd4c1ade, region_id=RegionOne) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:13.546792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7b67b-70b5-47b3-b622-fefd9c032643 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:13.547063 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d687c32-a368-4156-b551-6a1931d3cdc3 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:13.548350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7b67b-70b5-47b3-b622-fefd9c032643 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:13.556715 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 445/898] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Sep 9 16:19:13 2022] GET /identity/v3/endpoints?service_id=039b5bb67f954dd48328d423bd4c1ade&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:13.562080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7b67b-70b5-47b3-b622-fefd9c032643 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:13.594562 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac59fb-a5e7-44f2-b821-b875f56aca44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:13.595594 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac59fb-a5e7-44f2-b821-b875f56aca44 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:13.596077 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac59fb-a5e7-44f2-b821-b875f56aca44 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:13.596561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac59fb-a5e7-44f2-b821-b875f56aca44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:13.608030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac59fb-a5e7-44f2-b821-b875f56aca44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:13.609561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac59fb-a5e7-44f2-b821-b875f56aca44 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-38ac59fb-a5e7-44f2-b821-b875f56aca44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:13.627374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:13.628363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:13.629324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:13.629324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:13.629324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:13.629324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 454/899] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:13.635008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac59fb-a5e7-44f2-b821-b875f56aca44 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:13.679330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:13.685843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7b67b-70b5-47b3-b622-fefd9c032643 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:13.721442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c47f8-3c3d-41ec-a3d2-2ebcf9e5071d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:13.722051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c47f8-3c3d-41ec-a3d2-2ebcf9e5071d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:13.722362 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c47f8-3c3d-41ec-a3d2-2ebcf9e5071d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:13.722667 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c47f8-3c3d-41ec-a3d2-2ebcf9e5071d None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:13.723270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c47f8-3c3d-41ec-a3d2-2ebcf9e5071d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:13.724777 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c47f8-3c3d-41ec-a3d2-2ebcf9e5071d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:13.735635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 446/900] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Sep 9 16:19:13 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:19:13.762564 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac59fb-a5e7-44f2-b821-b875f56aca44 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:13.799804 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ba5bd-4c43-4856-ab12-385d2d843730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:13.800398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ba5bd-4c43-4856-ab12-385d2d843730 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:13.800676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ba5bd-4c43-4856-ab12-385d2d843730 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:13.800944 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ba5bd-4c43-4856-ab12-385d2d843730 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:13.801605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ba5bd-4c43-4856-ab12-385d2d843730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:13.801906 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ba5bd-4c43-4856-ab12-385d2d843730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:13.803528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ba5bd-4c43-4856-ab12-385d2d843730 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:13.811995 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:13.819676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 455/901] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:13.824919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c47f8-3c3d-41ec-a3d2-2ebcf9e5071d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:13.941415 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c47f8-3c3d-41ec-a3d2-2ebcf9e5071d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:13.976037 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb1a4a-0035-473a-a1ee-fa3121f521d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:13.976676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb1a4a-0035-473a-a1ee-fa3121f521d2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:13.976954 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb1a4a-0035-473a-a1ee-fa3121f521d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:13.977225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb1a4a-0035-473a-a1ee-fa3121f521d2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:13.985050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb1a4a-0035-473a-a1ee-fa3121f521d2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:13.986487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb1a4a-0035-473a-a1ee-fa3121f521d2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-5ccb1a4a-0035-473a-a1ee-fa3121f521d2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:19:13.997766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:13.998834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:13.999755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:13.999755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:13.999755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:19:13.999755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:19:13.999755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:19:13.999755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:19:13.999755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:19:13.999755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:19:13.999755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 16:19:13.999755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:13.999755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 447/902] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Sep 9 16:19:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:14.004633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ba5bd-4c43-4856-ab12-385d2d843730 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:14.118889 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ba5bd-4c43-4856-ab12-385d2d843730 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:14.153306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4df5ee-5994-4d07-9b50-64fc2fda5c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:14.154229 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4df5ee-5994-4d07-9b50-64fc2fda5c64 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:14.154746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4df5ee-5994-4d07-9b50-64fc2fda5c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:14.155194 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4df5ee-5994-4d07-9b50-64fc2fda5c64 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:14.156105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4df5ee-5994-4d07-9b50-64fc2fda5c64 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:14.156598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4df5ee-5994-4d07-9b50-64fc2fda5c64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:14.158274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4df5ee-5994-4d07-9b50-64fc2fda5c64 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:14.176714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 456/903] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Sep 9 16:19:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:14.182961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb1a4a-0035-473a-a1ee-fa3121f521d2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:14.204370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:14.288398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb1a4a-0035-473a-a1ee-fa3121f521d2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:14.323850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c87ea-7be8-42d2-bc0e-37826832597b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:14.324475 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c87ea-7be8-42d2-bc0e-37826832597b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:14.324756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c87ea-7be8-42d2-bc0e-37826832597b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:14.325025 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c87ea-7be8-42d2-bc0e-37826832597b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:14.397932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c87ea-7be8-42d2-bc0e-37826832597b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:14.399446 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c87ea-7be8-42d2-bc0e-37826832597b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:14.417958 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:14.503604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 448/904] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:19:14 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:19:14.509056 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4df5ee-5994-4d07-9b50-64fc2fda5c64 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:14.565874 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:14.614570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4df5ee-5994-4d07-9b50-64fc2fda5c64 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:14.649041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3bf794-2f52-4455-b088-4fa5f872ab70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:14.649594 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bf794-2f52-4455-b088-4fa5f872ab70 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:14.649866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bf794-2f52-4455-b088-4fa5f872ab70 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:14.650137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bf794-2f52-4455-b088-4fa5f872ab70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:14.660681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3bf794-2f52-4455-b088-4fa5f872ab70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:14.662367 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3bf794-2f52-4455-b088-4fa5f872ab70 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-7a3bf794-2f52-4455-b088-4fa5f872ab70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:14.678346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:14.679358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:14.680245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:14.680245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:14.680245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:14.680245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 457/905] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:14.688434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c87ea-7be8-42d2-bc0e-37826832597b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:14.803839 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c87ea-7be8-42d2-bc0e-37826832597b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:14.824286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6286c39-a4ee-4755-8891-6e23a9aeb685 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:14.824808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6286c39-a4ee-4755-8891-6e23a9aeb685 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:14.824808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6286c39-a4ee-4755-8891-6e23a9aeb685 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:14.826142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 458/906] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:19:14.827335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:14.831435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30624f-a538-44fa-8781-7cfb7f8a2718 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:14.831710 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30624f-a538-44fa-8781-7cfb7f8a2718 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:14.831980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30624f-a538-44fa-8781-7cfb7f8a2718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:14.845500 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30b96f-44c3-47be-9d25-159451802640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:14.846415 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30b96f-44c3-47be-9d25-159451802640 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:14.846850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30b96f-44c3-47be-9d25-159451802640 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:14.847287 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30b96f-44c3-47be-9d25-159451802640 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:14.848119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30b96f-44c3-47be-9d25-159451802640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:14.848426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30b96f-44c3-47be-9d25-159451802640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:14.849832 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30b96f-44c3-47be-9d25-159451802640 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:14.852777 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-4a30624f-a538-44fa-8781-7cfb7f8a2718 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:14.854203 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-4a30624f-a538-44fa-8781-7cfb7f8a2718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:14.861612 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 449/907] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:14.866977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30b96f-44c3-47be-9d25-159451802640 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:14.938397 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 459/908] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:19:14.977694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30b96f-44c3-47be-9d25-159451802640 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:15.011969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a27c5-2535-4ab5-9dbe-17b7b23a691a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:15.012855 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a27c5-2535-4ab5-9dbe-17b7b23a691a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:15.013294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a27c5-2535-4ab5-9dbe-17b7b23a691a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:15.013699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a27c5-2535-4ab5-9dbe-17b7b23a691a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:15.020902 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a27c5-2535-4ab5-9dbe-17b7b23a691a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:15.022627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a27c5-2535-4ab5-9dbe-17b7b23a691a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-433a27c5-2535-4ab5-9dbe-17b7b23a691a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:19:15.037280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:15.038440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:15.039585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:19:15.039585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:19:15.039585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:19:15.039585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:19:15.039585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:19:15.039585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:19:15.039585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:15.039585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:15.041877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 450/909] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:15.047879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30624f-a538-44fa-8781-7cfb7f8a2718 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:15.093380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:15.158742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30624f-a538-44fa-8781-7cfb7f8a2718 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:15.192623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9200b3a-723f-4bbe-8df5-61069b675a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:15.193245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9200b3a-723f-4bbe-8df5-61069b675a63 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:15.193519 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9200b3a-723f-4bbe-8df5-61069b675a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:15.193787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9200b3a-723f-4bbe-8df5-61069b675a63 None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:15.194471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9200b3a-723f-4bbe-8df5-61069b675a63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:15.194794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9200b3a-723f-4bbe-8df5-61069b675a63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:15.196213 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9200b3a-723f-4bbe-8df5-61069b675a63 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:15.208025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 460/910] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:19:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:15.213779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a27c5-2535-4ab5-9dbe-17b7b23a691a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:15.226256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9200b3a-723f-4bbe-8df5-61069b675a63 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:15.264853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:15.338767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:15.339162 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a27c5-2535-4ab5-9dbe-17b7b23a691a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:15.344496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9200b3a-723f-4bbe-8df5-61069b675a63 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:15.381725 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f77ec8-ca7c-4430-b902-851920cc904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:15.382337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f77ec8-ca7c-4430-b902-851920cc904e None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:15.382634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d499e-e5d1-4e8e-bcf1-94f7ab82b136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:15.382752 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f77ec8-ca7c-4430-b902-851920cc904e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:15.382929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f77ec8-ca7c-4430-b902-851920cc904e None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:15.383650 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f77ec8-ca7c-4430-b902-851920cc904e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:15.383932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d499e-e5d1-4e8e-bcf1-94f7ab82b136 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:15.384392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d499e-e5d1-4e8e-bcf1-94f7ab82b136 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:15.384845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d499e-e5d1-4e8e-bcf1-94f7ab82b136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:15.385017 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f77ec8-ca7c-4430-b902-851920cc904e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:15.385910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d499e-e5d1-4e8e-bcf1-94f7ab82b136 None admin] RBAC: Adding query filter params (role.id=4d4fab0f77f847fbb500c58e80767d9e, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=048b4abace244e48b6897daec0277e2f) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:15.386414 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d499e-e5d1-4e8e-bcf1-94f7ab82b136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:15.388123 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d499e-e5d1-4e8e-bcf1-94f7ab82b136 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:15.399193 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 451/911] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:19:15 2022] GET /identity/v3/role_assignments?user.id=048b4abace244e48b6897daec0277e2f&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=4d4fab0f77f847fbb500c58e80767d9e => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:15.417555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 461/912] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Sep 9 16:19:15 2022] POST /identity/v3/services => generated 241 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:15.716707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14d1bd-ffae-4a83-9909-763ba44f9690 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:15.717273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14d1bd-ffae-4a83-9909-763ba44f9690 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:15.717273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14d1bd-ffae-4a83-9909-763ba44f9690 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:15.718883 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 452/913] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:15.724239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3383edc-3eb4-467d-a1ff-e88b13783683 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:15.724522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3383edc-3eb4-467d-a1ff-e88b13783683 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:15.724796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3383edc-3eb4-467d-a1ff-e88b13783683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:15.743094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-e3383edc-3eb4-467d-a1ff-e88b13783683 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:15.744274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e3383edc-3eb4-467d-a1ff-e88b13783683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:15.819062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 462/914] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:16.034970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464c997-8459-4075-b12c-04afd4408b54 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:16.035176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464c997-8459-4075-b12c-04afd4408b54 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:16.035431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464c997-8459-4075-b12c-04afd4408b54 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:16.037061 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 453/915] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:16.042384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269e137-a13f-4ad3-9e5e-0621dbe004a8 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:16.042861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269e137-a13f-4ad3-9e5e-0621dbe004a8 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:16.042937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269e137-a13f-4ad3-9e5e-0621dbe004a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:16.061146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-5269e137-a13f-4ad3-9e5e-0621dbe004a8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:16.062633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5269e137-a13f-4ad3-9e5e-0621dbe004a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:16.094461 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464c997-8459-4075-b12c-04afd4408b54 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:16.140417 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 463/916] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:16.211907 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464c997-8459-4075-b12c-04afd4408b54 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:16.213397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:16.248044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c5b93-fed6-422f-be5a-63acf5d05d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:16.248675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c5b93-fed6-422f-be5a-63acf5d05d81 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:16.248956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c5b93-fed6-422f-be5a-63acf5d05d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:16.249214 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c5b93-fed6-422f-be5a-63acf5d05d81 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:16.249786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c5b93-fed6-422f-be5a-63acf5d05d81 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:16.251173 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c5b93-fed6-422f-be5a-63acf5d05d81 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-2a0c5b93-fed6-422f-be5a-63acf5d05d81 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:16.260350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:16.261261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:16.262130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 454/917] 10.222.0.23 () {56 vars in 1132 bytes} [Fri Sep 9 16:19:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:16.269572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269e137-a13f-4ad3-9e5e-0621dbe004a8 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:16.285564 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:16.381992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269e137-a13f-4ad3-9e5e-0621dbe004a8 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:16.405612 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c5b93-fed6-422f-be5a-63acf5d05d81 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:16.423456 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15318a-aa98-4c06-a91a-5806703c7046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:16.424532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15318a-aa98-4c06-a91a-5806703c7046 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:16.424806 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15318a-aa98-4c06-a91a-5806703c7046 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:16.425328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15318a-aa98-4c06-a91a-5806703c7046 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:16.425962 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15318a-aa98-4c06-a91a-5806703c7046 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:16.426522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15318a-aa98-4c06-a91a-5806703c7046 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:16.428785 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15318a-aa98-4c06-a91a-5806703c7046 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:16.436131 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 464/918] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Sep 9 16:19:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:16.441894 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15318a-aa98-4c06-a91a-5806703c7046 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:16.526219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:16.530170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c5b93-fed6-422f-be5a-63acf5d05d81 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:16.552498 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15318a-aa98-4c06-a91a-5806703c7046 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:16.569174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717f729-183b-4a70-81b7-ecb22ccd3de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:16.570117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717f729-183b-4a70-81b7-ecb22ccd3de7 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:16.570660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717f729-183b-4a70-81b7-ecb22ccd3de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:16.571124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717f729-183b-4a70-81b7-ecb22ccd3de7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:16.571909 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717f729-183b-4a70-81b7-ecb22ccd3de7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:16.573479 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717f729-183b-4a70-81b7-ecb22ccd3de7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-0717f729-183b-4a70-81b7-ecb22ccd3de7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:16.582555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:16.583627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:16.584484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:16.584484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:16.584484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:16.584484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:16.584484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:16.584484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:16.584484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:16.584484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:16.584484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:16.584484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:16.584484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:16.584484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:16.584484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 16:19:16.584484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:16.587264 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 455/919] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:19:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:16.593490 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717f729-183b-4a70-81b7-ecb22ccd3de7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:16.596597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837686d4-6059-4395-b87b-0c91fbc54487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:16.597559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837686d4-6059-4395-b87b-0c91fbc54487 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:16.598123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837686d4-6059-4395-b87b-0c91fbc54487 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:16.598614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837686d4-6059-4395-b87b-0c91fbc54487 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:16.599593 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837686d4-6059-4395-b87b-0c91fbc54487 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:16.600101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837686d4-6059-4395-b87b-0c91fbc54487 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:16.601693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837686d4-6059-4395-b87b-0c91fbc54487 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:16.612619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 465/920] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Sep 9 16:19:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:16.613812 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837686d4-6059-4395-b87b-0c91fbc54487 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:16.632275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:16.716038 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717f729-183b-4a70-81b7-ecb22ccd3de7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:16.723657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837686d4-6059-4395-b87b-0c91fbc54487 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:16.754279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42639e-2655-4064-a60b-6fd6ba2d5807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:16.754985 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42639e-2655-4064-a60b-6fd6ba2d5807 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:16.755250 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42639e-2655-4064-a60b-6fd6ba2d5807 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:16.755627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42639e-2655-4064-a60b-6fd6ba2d5807 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:16.756442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42639e-2655-4064-a60b-6fd6ba2d5807 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:16.756835 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42639e-2655-4064-a60b-6fd6ba2d5807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:16.758530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42639e-2655-4064-a60b-6fd6ba2d5807 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:16.758653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e88da1-94cb-45a1-b136-50ee948a2427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:16.759263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e88da1-94cb-45a1-b136-50ee948a2427 None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:16.759636 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e88da1-94cb-45a1-b136-50ee948a2427 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:16.759908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e88da1-94cb-45a1-b136-50ee948a2427 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:16.760672 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e88da1-94cb-45a1-b136-50ee948a2427 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:16.762092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e88da1-94cb-45a1-b136-50ee948a2427 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:16.766578 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 456/921] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:19:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:16.769073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42639e-2655-4064-a60b-6fd6ba2d5807 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:16.784665 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:16.794130 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 466/922] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:19:16 2022] POST /identity/v3/endpoints => generated 347 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:16.877354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:16.885087 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42639e-2655-4064-a60b-6fd6ba2d5807 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:16.928297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c5f68-9aba-45e8-9610-e7294b095137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:16.928930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c5f68-9aba-45e8-9610-e7294b095137 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:16.929205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c5f68-9aba-45e8-9610-e7294b095137 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:16.929471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c5f68-9aba-45e8-9610-e7294b095137 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:16.930111 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c5f68-9aba-45e8-9610-e7294b095137 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:16.930410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c5f68-9aba-45e8-9610-e7294b095137 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:16.931829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c5f68-9aba-45e8-9610-e7294b095137 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:16.938074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 457/923] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Sep 9 16:19:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:16.944531 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e88da1-94cb-45a1-b136-50ee948a2427 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:17.055248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e88da1-94cb-45a1-b136-50ee948a2427 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:17.091117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2d031-bdd6-4f4b-864c-2ab5d302c31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:17.091840 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2d031-bdd6-4f4b-864c-2ab5d302c31e None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:17.092127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2d031-bdd6-4f4b-864c-2ab5d302c31e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:17.092395 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2d031-bdd6-4f4b-864c-2ab5d302c31e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:17.093082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2d031-bdd6-4f4b-864c-2ab5d302c31e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:17.094541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2d031-bdd6-4f4b-864c-2ab5d302c31e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:17.119653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 467/924] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:19:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:17.649615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f24098-c329-43b0-81ee-1a13c9179c86 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:17.649962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f24098-c329-43b0-81ee-1a13c9179c86 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:17.650253 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f24098-c329-43b0-81ee-1a13c9179c86 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:17.651913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 458/925] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:17.657180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26410aa9-cf16-4819-80b6-5904258f7361 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:17.657463 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26410aa9-cf16-4819-80b6-5904258f7361 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:17.657736 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26410aa9-cf16-4819-80b6-5904258f7361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:17.676274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-26410aa9-cf16-4819-80b6-5904258f7361 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:17.677495 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-26410aa9-cf16-4819-80b6-5904258f7361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:17.687696 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128604c5-8d30-40b5-8c58-7942d456ec63 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:17.687991 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128604c5-8d30-40b5-8c58-7942d456ec63 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:17.688249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128604c5-8d30-40b5-8c58-7942d456ec63 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:17.689854 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 459/926] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:19:17.695325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13c6dd-155b-493d-8016-3be0a714da22 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:17.695602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13c6dd-155b-493d-8016-3be0a714da22 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:17.695872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13c6dd-155b-493d-8016-3be0a714da22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:17.716138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7f13c6dd-155b-493d-8016-3be0a714da22 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:17.717310 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7f13c6dd-155b-493d-8016-3be0a714da22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:17.729864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:17.759181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 468/927] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:17.793487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 460/928] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:19:18.026634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26410aa9-cf16-4819-80b6-5904258f7361 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:18.056178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13c6dd-155b-493d-8016-3be0a714da22 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:18.143162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26410aa9-cf16-4819-80b6-5904258f7361 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:18.157078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:18.177982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13c6dd-155b-493d-8016-3be0a714da22 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:18.183466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eefac7-7755-4c94-8067-d78d4aac852a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:18.184160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eefac7-7755-4c94-8067-d78d4aac852a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:18.184442 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eefac7-7755-4c94-8067-d78d4aac852a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:18.184714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eefac7-7755-4c94-8067-d78d4aac852a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:18.200699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eefac7-7755-4c94-8067-d78d4aac852a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-23eefac7-7755-4c94-8067-d78d4aac852a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:19:18.206100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:18.207334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:18.208330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:18.208330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:18.208330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:18.208330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:18.208330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:19:18.208330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:19:18.208330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:19:18.208330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:19:18.208330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:19:18.208330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:19:18.208330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 16:19:18.208330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:18.209239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 469/929] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Sep 9 16:19:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:18.215387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eefac7-7755-4c94-8067-d78d4aac852a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:18.222437 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c88a0-0a8c-48a2-a125-0c15365be60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:18.223352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c88a0-0a8c-48a2-a125-0c15365be60e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:18.223825 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c88a0-0a8c-48a2-a125-0c15365be60e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:18.224380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c88a0-0a8c-48a2-a125-0c15365be60e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:18.225156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c88a0-0a8c-48a2-a125-0c15365be60e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:18.226742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c88a0-0a8c-48a2-a125-0c15365be60e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-368c88a0-0a8c-48a2-a125-0c15365be60e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:18.235889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:18.236872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:18.237695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:18.237695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:18.237695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:18.237695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:18.237695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:18.237695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:18.237695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:18.237695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:18.237695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:18.237695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:18.237695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:18.237695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:18.237695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:18.237695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:18.240504 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 461/930] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Sep 9 16:19:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:18.246199 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c88a0-0a8c-48a2-a125-0c15365be60e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:18.263659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:18.325641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eefac7-7755-4c94-8067-d78d4aac852a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:18.364296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffbfcc-6744-4784-a45f-4eb8ec0bb1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:18.365039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffbfcc-6744-4784-a45f-4eb8ec0bb1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:18.365372 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffbfcc-6744-4784-a45f-4eb8ec0bb1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:18.365798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffbfcc-6744-4784-a45f-4eb8ec0bb1b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:18.366598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffbfcc-6744-4784-a45f-4eb8ec0bb1b7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:18.367002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffbfcc-6744-4784-a45f-4eb8ec0bb1b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:18.368377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffbfcc-6744-4784-a45f-4eb8ec0bb1b7 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:18.379489 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 470/931] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Sep 9 16:19:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:18.384826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffbfcc-6744-4784-a45f-4eb8ec0bb1b7 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:18.388042 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c88a0-0a8c-48a2-a125-0c15365be60e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:18.432718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a9642-65b1-45c8-8eaf-897403a220f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:18.434067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a9642-65b1-45c8-8eaf-897403a220f6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:18.434528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a9642-65b1-45c8-8eaf-897403a220f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:18.435323 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a9642-65b1-45c8-8eaf-897403a220f6 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:18.436361 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a9642-65b1-45c8-8eaf-897403a220f6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:18.436810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a9642-65b1-45c8-8eaf-897403a220f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:18.438615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a9642-65b1-45c8-8eaf-897403a220f6 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:18.445705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 462/932] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Sep 9 16:19:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:18.453435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a9642-65b1-45c8-8eaf-897403a220f6 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:18.476865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:18.509817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffbfcc-6744-4784-a45f-4eb8ec0bb1b7 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:18.511397 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:18.548958 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbace41-7081-450c-9c7f-cfd7ae8c46c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:18.549598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbace41-7081-450c-9c7f-cfd7ae8c46c2 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:18.549871 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbace41-7081-450c-9c7f-cfd7ae8c46c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:18.550138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbace41-7081-450c-9c7f-cfd7ae8c46c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:18.563826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbace41-7081-450c-9c7f-cfd7ae8c46c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:18.565957 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbace41-7081-450c-9c7f-cfd7ae8c46c2 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-dcbace41-7081-450c-9c7f-cfd7ae8c46c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:18.583640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:18.584601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:18.585496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:18.585496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:18.585496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:18.585496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 471/933] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:18.592890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbace41-7081-450c-9c7f-cfd7ae8c46c2 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:18.593025 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a9642-65b1-45c8-8eaf-897403a220f6 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:18.633649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70b011-ada7-4487-9c72-83060af99f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:18.634269 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70b011-ada7-4487-9c72-83060af99f50 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:18.634569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70b011-ada7-4487-9c72-83060af99f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:18.635011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70b011-ada7-4487-9c72-83060af99f50 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:18.635519 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70b011-ada7-4487-9c72-83060af99f50 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:18.635831 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70b011-ada7-4487-9c72-83060af99f50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:18.637239 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70b011-ada7-4487-9c72-83060af99f50 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:18.646543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 463/934] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Sep 9 16:19:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:18.649816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70b011-ada7-4487-9c72-83060af99f50 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:18.714957 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbace41-7081-450c-9c7f-cfd7ae8c46c2 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:18.741638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:18.750881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af4c0a-bf95-4a1a-8116-b71f4efaf5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:18.751557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af4c0a-bf95-4a1a-8116-b71f4efaf5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:18.751874 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af4c0a-bf95-4a1a-8116-b71f4efaf5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:18.752154 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af4c0a-bf95-4a1a-8116-b71f4efaf5e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:18.752800 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af4c0a-bf95-4a1a-8116-b71f4efaf5e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:18.753101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af4c0a-bf95-4a1a-8116-b71f4efaf5e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:18.754606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af4c0a-bf95-4a1a-8116-b71f4efaf5e3 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:18.768888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70b011-ada7-4487-9c72-83060af99f50 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:18.772724 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 472/935] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:18.773635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af4c0a-bf95-4a1a-8116-b71f4efaf5e3 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:18.790287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:18.804908 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c774a-717d-4824-a36d-baf968d1587a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:18.806204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c774a-717d-4824-a36d-baf968d1587a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:18.806696 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c774a-717d-4824-a36d-baf968d1587a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:18.807176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c774a-717d-4824-a36d-baf968d1587a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:18.808059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c774a-717d-4824-a36d-baf968d1587a None admin] RBAC: Adding query filter params (interface=public, service_id=e59850358b8d4773a90a46017a9fc48e, region_id=RegionOne) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:18.808552 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c774a-717d-4824-a36d-baf968d1587a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:18.810113 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c774a-717d-4824-a36d-baf968d1587a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:18.820826 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 464/936] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Sep 9 16:19:18 2022] GET /identity/v3/endpoints?service_id=e59850358b8d4773a90a46017a9fc48e&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:18.822332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c774a-717d-4824-a36d-baf968d1587a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:18.882845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af4c0a-bf95-4a1a-8116-b71f4efaf5e3 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:18.919679 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367db72e-e127-457a-9f4b-1634135c212a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:18.920439 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367db72e-e127-457a-9f4b-1634135c212a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:18.920724 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367db72e-e127-457a-9f4b-1634135c212a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:18.920998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367db72e-e127-457a-9f4b-1634135c212a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:18.924735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c774a-717d-4824-a36d-baf968d1587a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:18.929059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367db72e-e127-457a-9f4b-1634135c212a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:18.930589 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367db72e-e127-457a-9f4b-1634135c212a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-367db72e-e127-457a-9f4b-1634135c212a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:19:18.944011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:18.945012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:18.945955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:18.945955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:18.945955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:19:18.945955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:19:18.945955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:19:18.945955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:19:18.945955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:19:18.945955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:19:18.945955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 16:19:18.945955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:18.949949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 473/937] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Sep 9 16:19:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:18.951038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367db72e-e127-457a-9f4b-1634135c212a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:18.965240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120bc26-943e-47db-a798-222de068f202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:18.966181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120bc26-943e-47db-a798-222de068f202 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:18.966816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120bc26-943e-47db-a798-222de068f202 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:18.967275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120bc26-943e-47db-a798-222de068f202 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:18.968077 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120bc26-943e-47db-a798-222de068f202 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:18.969636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120bc26-943e-47db-a798-222de068f202 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:18.982670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 465/938] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Sep 9 16:19:18 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:19:19.040568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:19.071433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367db72e-e127-457a-9f4b-1634135c212a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:19.109209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044c85a-88c1-460e-9ff6-d234ef1232a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:19.109902 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044c85a-88c1-460e-9ff6-d234ef1232a4 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:19.110181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044c85a-88c1-460e-9ff6-d234ef1232a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:19.110513 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044c85a-88c1-460e-9ff6-d234ef1232a4 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:19.111275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044c85a-88c1-460e-9ff6-d234ef1232a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:19.111638 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044c85a-88c1-460e-9ff6-d234ef1232a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:19.113105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044c85a-88c1-460e-9ff6-d234ef1232a4 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:19.124645 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59002cab-8d07-45c5-b3e1-b7d78374a724 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:19.124986 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59002cab-8d07-45c5-b3e1-b7d78374a724 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:19.125200 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59002cab-8d07-45c5-b3e1-b7d78374a724 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:19.126879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 466/939] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:19.132767 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad5b8b-b0a1-4d4c-9745-dd6cec5f73c8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:19.133159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad5b8b-b0a1-4d4c-9745-dd6cec5f73c8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:19.133338 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad5b8b-b0a1-4d4c-9745-dd6cec5f73c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:19.134110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 474/940] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Sep 9 16:19:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:19.141423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:19.141744 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044c85a-88c1-460e-9ff6-d234ef1232a4 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:19.153932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c5ad5b8b-b0a1-4d4c-9745-dd6cec5f73c8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:19.155213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c5ad5b8b-b0a1-4d4c-9745-dd6cec5f73c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:19.244730 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 467/941] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:19:19.260763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044c85a-88c1-460e-9ff6-d234ef1232a4 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:19.291535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:19.296401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67dc57-06bd-4dd8-99c3-f0563a908a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:19.297046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67dc57-06bd-4dd8-99c3-f0563a908a3a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:19.297319 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67dc57-06bd-4dd8-99c3-f0563a908a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:19.297582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67dc57-06bd-4dd8-99c3-f0563a908a3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:19.308587 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67dc57-06bd-4dd8-99c3-f0563a908a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:19.310076 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67dc57-06bd-4dd8-99c3-f0563a908a3a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:19.325663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2294f-a6ef-43b2-8404-b9a17c64d372 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-4a67dc57-06bd-4dd8-99c3-f0563a908a3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:19.325876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:19.326773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:19.327659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:19.327659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:19.327659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:19.327825 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2294f-a6ef-43b2-8404-b9a17c64d372 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:19.327825 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2294f-a6ef-43b2-8404-b9a17c64d372 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:19.327956 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 475/942] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:19.328025 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 468/943] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:19:19.332354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67dc57-06bd-4dd8-99c3-f0563a908a3a None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:19.332922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38390aab-dc03-46ea-bcf2-c79fea8abc27 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:19.333435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38390aab-dc03-46ea-bcf2-c79fea8abc27 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:19.333718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38390aab-dc03-46ea-bcf2-c79fea8abc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:19.352605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-38390aab-dc03-46ea-bcf2-c79fea8abc27 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:19.353849 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-38390aab-dc03-46ea-bcf2-c79fea8abc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:19.424443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 469/944] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:19.444820 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67dc57-06bd-4dd8-99c3-f0563a908a3a None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:19.480141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b826f-0991-43fe-835e-8120dca7778c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:19.480752 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b826f-0991-43fe-835e-8120dca7778c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:19.481023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b826f-0991-43fe-835e-8120dca7778c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:19.481286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b826f-0991-43fe-835e-8120dca7778c None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:19.481927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b826f-0991-43fe-835e-8120dca7778c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:19.482222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b826f-0991-43fe-835e-8120dca7778c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:19.483692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b826f-0991-43fe-835e-8120dca7778c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:19.496286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 476/945] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:19.502926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38390aab-dc03-46ea-bcf2-c79fea8abc27 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:19.521572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b826f-0991-43fe-835e-8120dca7778c None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:19.569275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:19.582534 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:19.626383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b826f-0991-43fe-835e-8120dca7778c None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:19.633738 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38390aab-dc03-46ea-bcf2-c79fea8abc27 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:19.662168 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de22fe-deb8-44c5-a9ff-3ce2ef8cdfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:19.663304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de22fe-deb8-44c5-a9ff-3ce2ef8cdfcc None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:19.663850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de22fe-deb8-44c5-a9ff-3ce2ef8cdfcc None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:19.664380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de22fe-deb8-44c5-a9ff-3ce2ef8cdfcc None admin] PATH_INFO: `/v3/services/compute` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:19.665476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de22fe-deb8-44c5-a9ff-3ce2ef8cdfcc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:19.667355 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de22fe-deb8-44c5-a9ff-3ce2ef8cdfcc None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-c8de22fe-deb8-44c5-a9ff-3ce2ef8cdfcc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:19.676769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:19.678055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:19.679125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:19.679125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:19.679125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:19.679125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:19.679125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:19.679125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:19.679125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:19.679125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:19.679125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:19.679125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:19.679125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:19.679125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:19.679125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 16:19:19.679125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:19.680582 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b304d7-0dd7-46d4-811b-32a111ea6cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:19.681371 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b304d7-0dd7-46d4-811b-32a111ea6cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:19.681778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b304d7-0dd7-46d4-811b-32a111ea6cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:19.682160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b304d7-0dd7-46d4-811b-32a111ea6cc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:19.685412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 477/946] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:19.686209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de22fe-deb8-44c5-a9ff-3ce2ef8cdfcc None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:19.691201 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b304d7-0dd7-46d4-811b-32a111ea6cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:19.692732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b304d7-0dd7-46d4-811b-32a111ea6cc2 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c7b304d7-0dd7-46d4-811b-32a111ea6cc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:19:19.706894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:19.707798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:19.708723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:19:19.708723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:19:19.708723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:19:19.708723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:19:19.708723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:19:19.708723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:19:19.708723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:19.708723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:19.710636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 470/947] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:19.711621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b304d7-0dd7-46d4-811b-32a111ea6cc2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:19.795487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de22fe-deb8-44c5-a9ff-3ce2ef8cdfcc None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:19.804713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:19.812932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b304d7-0dd7-46d4-811b-32a111ea6cc2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:19.831071 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5168ed-2ac5-4182-a805-5dbd1adb1606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:19.831685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5168ed-2ac5-4182-a805-5dbd1adb1606 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:19.831961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5168ed-2ac5-4182-a805-5dbd1adb1606 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:19.832227 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5168ed-2ac5-4182-a805-5dbd1adb1606 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:19.832851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5168ed-2ac5-4182-a805-5dbd1adb1606 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:19.833150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5168ed-2ac5-4182-a805-5dbd1adb1606 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:19.834576 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5168ed-2ac5-4182-a805-5dbd1adb1606 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:19.837350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:19.839721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 478/948] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Sep 9 16:19:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:19.840424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5168ed-2ac5-4182-a805-5dbd1adb1606 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:19.855519 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcb39c-9e7c-4acd-869d-711cc8a71737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:19.856133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcb39c-9e7c-4acd-869d-711cc8a71737 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:19.856399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcb39c-9e7c-4acd-869d-711cc8a71737 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:19.856672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcb39c-9e7c-4acd-869d-711cc8a71737 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:19.857309 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcb39c-9e7c-4acd-869d-711cc8a71737 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:19.858774 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcb39c-9e7c-4acd-869d-711cc8a71737 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:19.861394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: WARNING keystone.server.flask.common [None req-d9fcb39c-9e7c-4acd-869d-711cc8a71737 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 16:19:19.879618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.sql.core [None req-d9fcb39c-9e7c-4acd-869d-711cc8a71737 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 16:19:19.879618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 16:19:19.879618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [parameters: {'id': 'dbb5218e787446eeae4b753228309d24', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 16:19:19.879618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81511) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 16:19:19.895506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-d9fcb39c-9e7c-4acd-869d-711cc8a71737 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dbb5218e787446eeae4b753228309d24.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dbb5218e787446eeae4b753228309d24. Sep 09 16:19:19.895506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:19.895506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 16:19:19.895506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 16:19:19.895506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 16:19:19.895506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 16:19:19.895506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 16:19:19.895506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 16:19:19.895506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 16:19:19.895506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application conn.query(q) Sep 09 16:19:19.895506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 16:19:19.895506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 16:19:19.895506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application result.read() Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 09 16:19:19.896520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application next(self.gen) Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application yield resource Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application next(self.gen) Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise self.value Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application session.commit() Sep 09 16:19:19.897649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self.session.flush() Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application compat.raise_( Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 16:19:19.898797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 16:19:19.899865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application conn.query(q) Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application result.read() Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 16:19:19.900803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [parameters: {'id': 'dbb5218e787446eeae4b753228309d24', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:19.901780 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:19.902772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:19.903889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:19.903889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 09 16:19:19.903889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 09 16:19:19.903889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:19.903889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:19.903889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 09 16:19:19.903889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 09 16:19:19.903889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dbb5218e787446eeae4b753228309d24. Sep 09 16:19:19.903889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:19.903889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 471/949] 10.222.0.23 () {60 vars in 1140 bytes} [Fri Sep 9 16:19:19 2022] POST /identity/v3/projects => generated 262 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:19.903889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcb39c-9e7c-4acd-869d-711cc8a71737 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:19.956822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5168ed-2ac5-4182-a805-5dbd1adb1606 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:19.992759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048847a5-b5b8-4141-bacf-b408de279dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:19.993436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048847a5-b5b8-4141-bacf-b408de279dae None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:19.993712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048847a5-b5b8-4141-bacf-b408de279dae None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:19.993975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048847a5-b5b8-4141-bacf-b408de279dae None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:19.994813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048847a5-b5b8-4141-bacf-b408de279dae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:19.995221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048847a5-b5b8-4141-bacf-b408de279dae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:19.996633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048847a5-b5b8-4141-bacf-b408de279dae None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:20.003559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcb39c-9e7c-4acd-869d-711cc8a71737 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:20.014454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 479/950] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:19:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:20.014454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048847a5-b5b8-4141-bacf-b408de279dae None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:20.043096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831230b-df03-4619-a862-7d0c7e1d894f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:20.043701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831230b-df03-4619-a862-7d0c7e1d894f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:20.043992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831230b-df03-4619-a862-7d0c7e1d894f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:20.044237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831230b-df03-4619-a862-7d0c7e1d894f None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:20.044860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831230b-df03-4619-a862-7d0c7e1d894f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:20.045156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831230b-df03-4619-a862-7d0c7e1d894f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:20.045494 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:20.046749 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831230b-df03-4619-a862-7d0c7e1d894f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:20.054250 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 472/951] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Sep 9 16:19:19 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:20.054947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831230b-df03-4619-a862-7d0c7e1d894f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:20.122724 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:20.149644 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048847a5-b5b8-4141-bacf-b408de279dae None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:20.187007 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831230b-df03-4619-a862-7d0c7e1d894f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:20.199334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062dd96f-b765-40cc-bba2-94437675a2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:20.200937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062dd96f-b765-40cc-bba2-94437675a2ec None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:20.201528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062dd96f-b765-40cc-bba2-94437675a2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:20.201978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062dd96f-b765-40cc-bba2-94437675a2ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:20.211775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062dd96f-b765-40cc-bba2-94437675a2ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:20.213934 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062dd96f-b765-40cc-bba2-94437675a2ec None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:20.226215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5e517-fe41-4f39-ad94-78a795b62f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:20.227038 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5e517-fe41-4f39-ad94-78a795b62f89 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:20.227377 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5e517-fe41-4f39-ad94-78a795b62f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:20.227711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5e517-fe41-4f39-ad94-78a795b62f89 None admin] PATH_INFO: `/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/048b4abace244e48b6897daec0277e2f/roles/4d4fab0f77f847fbb500c58e80767d9e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-062dd96f-b765-40cc-bba2-94437675a2ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:19:20.229787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:20.230872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:20.231788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:19:20.231788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:19:20.231788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:19:20.231788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:19:20.231788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:19:20.231788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:19:20.231788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:20.231788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:20.232206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5e517-fe41-4f39-ad94-78a795b62f89 None admin] RBAC: Authorizing `identity:create_grant(project_id=253ff30a56444df28d44c4b7eaadb276, user_id=048b4abace244e48b6897daec0277e2f, role_id=4d4fab0f77f847fbb500c58e80767d9e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:20.232206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5e517-fe41-4f39-ad94-78a795b62f89 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:20.234821 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 480/952] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:20.240804 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062dd96f-b765-40cc-bba2-94437675a2ec None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:20.253394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 473/953] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:19:20 2022] PUT /identity/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/048b4abace244e48b6897daec0277e2f/roles/4d4fab0f77f847fbb500c58e80767d9e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:19:20.351209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:20.362255 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062dd96f-b765-40cc-bba2-94437675a2ec None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:20.398907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2c2b4-7347-42d3-a0d9-7a6d917a202c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:20.399764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2c2b4-7347-42d3-a0d9-7a6d917a202c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:20.400045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2c2b4-7347-42d3-a0d9-7a6d917a202c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:20.400317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2c2b4-7347-42d3-a0d9-7a6d917a202c None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:20.400998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2c2b4-7347-42d3-a0d9-7a6d917a202c None admin] RBAC: Adding query filter params (name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:20.401350 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2c2b4-7347-42d3-a0d9-7a6d917a202c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:20.402864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2c2b4-7347-42d3-a0d9-7a6d917a202c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:20.424080 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 481/954] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Sep 9 16:19:20 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:21.193211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d2bdd-1a22-4515-a1b7-ccb5fd0e26dd None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:21.193746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d2bdd-1a22-4515-a1b7-ccb5fd0e26dd None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:21.193746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d2bdd-1a22-4515-a1b7-ccb5fd0e26dd None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:21.195464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 474/955] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:21.201213 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e80202-ea86-4999-a4a9-d17deb1c3025 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:21.201496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e80202-ea86-4999-a4a9-d17deb1c3025 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:21.201763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e80202-ea86-4999-a4a9-d17deb1c3025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:21.220312 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-36e80202-ea86-4999-a4a9-d17deb1c3025 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:21.221495 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-36e80202-ea86-4999-a4a9-d17deb1c3025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:21.294631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 482/956] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:21.555864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d2bdd-1a22-4515-a1b7-ccb5fd0e26dd None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:21.660259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d2bdd-1a22-4515-a1b7-ccb5fd0e26dd None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:21.662633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:21.696112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329fecd-f5ff-4989-8832-62819f5d00dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:21.696745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329fecd-f5ff-4989-8832-62819f5d00dd None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:21.697023 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329fecd-f5ff-4989-8832-62819f5d00dd None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:21.697292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329fecd-f5ff-4989-8832-62819f5d00dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:21.697876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3329fecd-f5ff-4989-8832-62819f5d00dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:21.699277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3329fecd-f5ff-4989-8832-62819f5d00dd None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-3329fecd-f5ff-4989-8832-62819f5d00dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:21.708417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:21.709703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:21.710679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 475/957] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Sep 9 16:19:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:21.715582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e80202-ea86-4999-a4a9-d17deb1c3025 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:21.801286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:21.820358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e80202-ea86-4999-a4a9-d17deb1c3025 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:21.856757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73495390-b0bd-42de-bcef-a6e9c6dee319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:21.857432 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73495390-b0bd-42de-bcef-a6e9c6dee319 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:21.857714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73495390-b0bd-42de-bcef-a6e9c6dee319 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:21.857987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73495390-b0bd-42de-bcef-a6e9c6dee319 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:21.858648 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73495390-b0bd-42de-bcef-a6e9c6dee319 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:21.858949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73495390-b0bd-42de-bcef-a6e9c6dee319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:21.860314 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73495390-b0bd-42de-bcef-a6e9c6dee319 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:21.869968 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 483/958] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Sep 9 16:19:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:21.871367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329fecd-f5ff-4989-8832-62819f5d00dd None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:21.971795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329fecd-f5ff-4989-8832-62819f5d00dd None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:22.010234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766846d5-a5ce-4212-aac1-7b778d98aa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:22.010877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766846d5-a5ce-4212-aac1-7b778d98aa95 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:22.011154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766846d5-a5ce-4212-aac1-7b778d98aa95 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:22.011423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766846d5-a5ce-4212-aac1-7b778d98aa95 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:22.012042 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766846d5-a5ce-4212-aac1-7b778d98aa95 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:22.012340 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766846d5-a5ce-4212-aac1-7b778d98aa95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:22.013714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766846d5-a5ce-4212-aac1-7b778d98aa95 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:22.019109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 476/959] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Sep 9 16:19:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:22.024875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73495390-b0bd-42de-bcef-a6e9c6dee319 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:22.139420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73495390-b0bd-42de-bcef-a6e9c6dee319 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:22.176032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6d427-a411-4cca-9125-65b24ca3900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:22.177177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6d427-a411-4cca-9125-65b24ca3900e None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:22.177668 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6d427-a411-4cca-9125-65b24ca3900e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:22.178133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6d427-a411-4cca-9125-65b24ca3900e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:22.179103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6d427-a411-4cca-9125-65b24ca3900e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:22.180834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6d427-a411-4cca-9125-65b24ca3900e None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:22.207569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 484/960] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:19:22 2022] POST /identity/v3/endpoints => generated 349 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:22.296266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870f7a4-f9ef-4198-9246-dd02105ce25a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:22.296834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870f7a4-f9ef-4198-9246-dd02105ce25a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:22.296834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870f7a4-f9ef-4198-9246-dd02105ce25a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:22.298482 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 477/961] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:22.304395 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0e906-1f85-43d1-a87a-6461139e8867 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:22.304734 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0e906-1f85-43d1-a87a-6461139e8867 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:22.305041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0e906-1f85-43d1-a87a-6461139e8867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:22.311880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:22.330281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-6eb0e906-1f85-43d1-a87a-6461139e8867 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:22.331939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-6eb0e906-1f85-43d1-a87a-6461139e8867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:22.414982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 485/962] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:22.537643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0319e-ec6b-4400-bfdc-6de82fec1b5d None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:22.537982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0319e-ec6b-4400-bfdc-6de82fec1b5d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:22.538189 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0319e-ec6b-4400-bfdc-6de82fec1b5d None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:22.539831 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 478/963] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:22.545041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b4d50-e6d6-4de1-9b02-c4c3ec23b862 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:22.545330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b4d50-e6d6-4de1-9b02-c4c3ec23b862 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:22.545600 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b4d50-e6d6-4de1-9b02-c4c3ec23b862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:22.564468 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-598b4d50-e6d6-4de1-9b02-c4c3ec23b862 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:22.566065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-598b4d50-e6d6-4de1-9b02-c4c3ec23b862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:22.639867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 486/964] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:22.693054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75345010-252f-49b9-a78c-1c9e99388d33 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:22.693231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75345010-252f-49b9-a78c-1c9e99388d33 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:22.693501 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75345010-252f-49b9-a78c-1c9e99388d33 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:22.695181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 479/965] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:22.700411 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677191b-2f51-4e19-a65a-e87f34e4674c None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:22.700697 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677191b-2f51-4e19-a65a-e87f34e4674c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:22.700964 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677191b-2f51-4e19-a65a-e87f34e4674c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:22.716680 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75345010-252f-49b9-a78c-1c9e99388d33 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:22.721600 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-a677191b-2f51-4e19-a65a-e87f34e4674c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:22.722826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a677191b-2f51-4e19-a65a-e87f34e4674c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:22.755465 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:22.781618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:22.797875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 487/966] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:22.838377 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75345010-252f-49b9-a78c-1c9e99388d33 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:22.873884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba3c60-ebf1-4d20-b537-134e5a3d8a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:22.874942 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba3c60-ebf1-4d20-b537-134e5a3d8a76 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:22.875419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba3c60-ebf1-4d20-b537-134e5a3d8a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:22.875881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba3c60-ebf1-4d20-b537-134e5a3d8a76 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:22.876768 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba3c60-ebf1-4d20-b537-134e5a3d8a76 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:22.878381 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba3c60-ebf1-4d20-b537-134e5a3d8a76 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:22.895262 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 480/967] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Sep 9 16:19:22 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:22.937070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677191b-2f51-4e19-a65a-e87f34e4674c None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:23.045293 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677191b-2f51-4e19-a65a-e87f34e4674c None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:23.071514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba3c60-ebf1-4d20-b537-134e5a3d8a76 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:23.113836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13094125-1854-482a-9255-9c9fceda135c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:23.115081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13094125-1854-482a-9255-9c9fceda135c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:23.115504 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13094125-1854-482a-9255-9c9fceda135c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:23.115780 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13094125-1854-482a-9255-9c9fceda135c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:23.131213 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13094125-1854-482a-9255-9c9fceda135c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-13094125-1854-482a-9255-9c9fceda135c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:19:23.136088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:23.137045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:23.137897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:23.137897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:23.137897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:23.137897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:23.137897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:19:23.137897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:19:23.137897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:19:23.137897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:19:23.137897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:19:23.137897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:19:23.137897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 16:19:23.137897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:23.137897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 488/968] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Sep 9 16:19:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:23.144820 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13094125-1854-482a-9255-9c9fceda135c None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:23.194959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:23.207134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba3c60-ebf1-4d20-b537-134e5a3d8a76 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:23.210955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:23.245228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74614cf6-d974-4629-a22d-6712a5172bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:23.245228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74614cf6-d974-4629-a22d-6712a5172bff None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:23.245549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74614cf6-d974-4629-a22d-6712a5172bff None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:23.245796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74614cf6-d974-4629-a22d-6712a5172bff None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:23.246567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74614cf6-d974-4629-a22d-6712a5172bff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:23.248112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74614cf6-d974-4629-a22d-6712a5172bff None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:23.264470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.sql.core [None req-74614cf6-d974-4629-a22d-6712a5172bff None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 16:19:23.264470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 16:19:23.264470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [parameters: {'user_id': '91b53095c0af4f8cb5998dacaab6faaf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 16:19:23.264470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81511) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-74614cf6-d974-4629-a22d-6712a5172bff None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application conn.query(q) Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application result.read() Sep 09 16:19:23.271756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 16:19:23.272685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application next(self.gen) Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application yield resource Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application next(self.gen) Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise self.value Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application session.commit() Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 16:19:23.273634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self.session.flush() Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application compat.raise_( Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 16:19:23.274533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 16:19:23.275432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application conn.query(q) Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application result.read() Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 16:19:23.276274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [parameters: {'user_id': '91b53095c0af4f8cb5998dacaab6faaf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:23.277188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 09 16:19:23.281040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 09 16:19:23.283289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 09 16:19:23.283289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 09 16:19:23.283289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 16:19:23.283289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 16:19:23.283289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 16:19:23.283289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:23.283289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 481/969] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Sep 9 16:19:23 2022] POST /identity/v3/users => generated 160 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:23.283289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74614cf6-d974-4629-a22d-6712a5172bff None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:23.293879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13094125-1854-482a-9255-9c9fceda135c None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:23.335550 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e95ae0-8373-4c54-bfdc-a213587d55e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:23.336645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e95ae0-8373-4c54-bfdc-a213587d55e5 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:23.337460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e95ae0-8373-4c54-bfdc-a213587d55e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:23.337926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e95ae0-8373-4c54-bfdc-a213587d55e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:23.338804 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e95ae0-8373-4c54-bfdc-a213587d55e5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:23.339279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e95ae0-8373-4c54-bfdc-a213587d55e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:23.341340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e95ae0-8373-4c54-bfdc-a213587d55e5 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:23.351800 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 489/970] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Sep 9 16:19:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:23.357707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e95ae0-8373-4c54-bfdc-a213587d55e5 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:23.396115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74614cf6-d974-4629-a22d-6712a5172bff None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:23.438250 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b6852-920d-4849-af20-3512fb602f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:23.439200 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6852-920d-4849-af20-3512fb602f60 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:23.439741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6852-920d-4849-af20-3512fb602f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:23.440568 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6852-920d-4849-af20-3512fb602f60 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:23.448708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b6852-920d-4849-af20-3512fb602f60 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:23.450950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b6852-920d-4849-af20-3512fb602f60 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-098b6852-920d-4849-af20-3512fb602f60 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:19:23.463318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:23.464301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:23.465243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:23.465243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:23.465243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:19:23.465243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:19:23.465243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:19:23.465243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:19:23.465243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:19:23.465243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:19:23.465243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 16:19:23.465243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:23.469499 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 482/971] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:19:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:23.470462 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b6852-920d-4849-af20-3512fb602f60 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:23.475771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e95ae0-8373-4c54-bfdc-a213587d55e5 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:23.484732 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:23.503678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:23.523586 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19db3e-057b-42e7-9cbd-5c67a803741f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:23.524996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19db3e-057b-42e7-9cbd-5c67a803741f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:23.525453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19db3e-057b-42e7-9cbd-5c67a803741f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:23.526246 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19db3e-057b-42e7-9cbd-5c67a803741f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:23.541314 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19db3e-057b-42e7-9cbd-5c67a803741f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:23.543926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19db3e-057b-42e7-9cbd-5c67a803741f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-fd19db3e-057b-42e7-9cbd-5c67a803741f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:23.563634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:23.564643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:23.565897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:23.565897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:23.565897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:23.573067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 490/972] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:23.574167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19db3e-057b-42e7-9cbd-5c67a803741f None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:23.592181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b6852-920d-4849-af20-3512fb602f60 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:23.626758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c03a4-8c48-44cc-b238-f8488976e258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:23.627519 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c03a4-8c48-44cc-b238-f8488976e258 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:23.627894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c03a4-8c48-44cc-b238-f8488976e258 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:23.628257 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c03a4-8c48-44cc-b238-f8488976e258 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:23.629058 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c03a4-8c48-44cc-b238-f8488976e258 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:23.629454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c03a4-8c48-44cc-b238-f8488976e258 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:23.631074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c03a4-8c48-44cc-b238-f8488976e258 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:23.671673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 483/973] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Sep 9 16:19:23 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:23.698257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19db3e-057b-42e7-9cbd-5c67a803741f None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:23.736473 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19796c1c-393a-439b-bce3-f7a2eff7b258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:23.737881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19796c1c-393a-439b-bce3-f7a2eff7b258 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:23.738379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19796c1c-393a-439b-bce3-f7a2eff7b258 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:23.738817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19796c1c-393a-439b-bce3-f7a2eff7b258 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:23.739712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19796c1c-393a-439b-bce3-f7a2eff7b258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:23.740173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19796c1c-393a-439b-bce3-f7a2eff7b258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:23.741716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19796c1c-393a-439b-bce3-f7a2eff7b258 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:23.756113 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 491/974] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:23.762165 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c03a4-8c48-44cc-b238-f8488976e258 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:23.843294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:23.876974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c03a4-8c48-44cc-b238-f8488976e258 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:23.914691 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3245f-2567-404e-b367-f353df1d2f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:23.915381 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3245f-2567-404e-b367-f353df1d2f69 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:23.915627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3245f-2567-404e-b367-f353df1d2f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:23.915898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3245f-2567-404e-b367-f353df1d2f69 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:23.924689 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3245f-2567-404e-b367-f353df1d2f69 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:23.926166 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3245f-2567-404e-b367-f353df1d2f69 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-01c3245f-2567-404e-b367-f353df1d2f69 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:19:23.938273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:23.939417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:23.940722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:23.940722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:23.940722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:19:23.940722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:19:23.940722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:19:23.940722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:19:23.940722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:19:23.940722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:19:23.940722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 16:19:23.940722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:23.940722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 484/975] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Sep 9 16:19:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:23.946065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19796c1c-393a-439b-bce3-f7a2eff7b258 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:23.967863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:24.053168 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19796c1c-393a-439b-bce3-f7a2eff7b258 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:24.092455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c84a3e-f122-4848-ac91-c9dd1688f654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:24.093236 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c84a3e-f122-4848-ac91-c9dd1688f654 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:24.093653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c84a3e-f122-4848-ac91-c9dd1688f654 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:24.093971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c84a3e-f122-4848-ac91-c9dd1688f654 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:24.094945 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c84a3e-f122-4848-ac91-c9dd1688f654 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:24.095348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c84a3e-f122-4848-ac91-c9dd1688f654 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:24.096858 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c84a3e-f122-4848-ac91-c9dd1688f654 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:24.118549 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 492/976] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Sep 9 16:19:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:24.120151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3245f-2567-404e-b367-f353df1d2f69 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:24.220938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3245f-2567-404e-b367-f353df1d2f69 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:24.250106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:24.256218 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5b017-bca5-43de-9d2a-21737702b4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:24.256834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5b017-bca5-43de-9d2a-21737702b4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:24.257111 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5b017-bca5-43de-9d2a-21737702b4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:24.257380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5b017-bca5-43de-9d2a-21737702b4d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:24.329326 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5b017-bca5-43de-9d2a-21737702b4d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:24.330858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5b017-bca5-43de-9d2a-21737702b4d1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:24.428287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c723d49-8b3b-426c-a5cb-37a85acc8de5 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:24.428566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c723d49-8b3b-426c-a5cb-37a85acc8de5 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:24.428852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c723d49-8b3b-426c-a5cb-37a85acc8de5 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:24.430369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 493/977] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:19:24.437736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 485/978] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:19:24 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:19:24.439080 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b6778-b940-49d3-b4e6-e45d1db24a99 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:24.439358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b6778-b940-49d3-b4e6-e45d1db24a99 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:24.439491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5b017-bca5-43de-9d2a-21737702b4d1 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:24.439582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b6778-b940-49d3-b4e6-e45d1db24a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:24.458129 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d42b6778-b940-49d3-b4e6-e45d1db24a99 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:24.459348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d42b6778-b940-49d3-b4e6-e45d1db24a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:24.469853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:24.536054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 494/979] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:19:24.558957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5b017-bca5-43de-9d2a-21737702b4d1 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:24.595263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4493f6-3f1a-4445-b477-3879a8055768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:24.595899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4493f6-3f1a-4445-b477-3879a8055768 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:24.596136 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4493f6-3f1a-4445-b477-3879a8055768 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:24.596435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4493f6-3f1a-4445-b477-3879a8055768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:24.606768 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4493f6-3f1a-4445-b477-3879a8055768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:24.608223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4493f6-3f1a-4445-b477-3879a8055768 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-9a4493f6-3f1a-4445-b477-3879a8055768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:24.622747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:24.623685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:24.624662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:24.624662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:24.624662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:24.625371 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 486/980] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:24.629604 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b6778-b940-49d3-b4e6-e45d1db24a99 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:24.638269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:24.727284 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b6778-b940-49d3-b4e6-e45d1db24a99 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:24.760107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dfbf1-096c-4890-8936-b122bcf45582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:24.760727 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dfbf1-096c-4890-8936-b122bcf45582 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:24.760966 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dfbf1-096c-4890-8936-b122bcf45582 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:24.761255 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dfbf1-096c-4890-8936-b122bcf45582 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:24.761882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60dfbf1-096c-4890-8936-b122bcf45582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:24.762182 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60dfbf1-096c-4890-8936-b122bcf45582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:24.763577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60dfbf1-096c-4890-8936-b122bcf45582 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:24.775073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 495/981] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:24.780660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4493f6-3f1a-4445-b477-3879a8055768 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:24.796773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dfbf1-096c-4890-8936-b122bcf45582 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:24.868462 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:24.872383 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:24.893367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4493f6-3f1a-4445-b477-3879a8055768 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:24.907149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dfbf1-096c-4890-8936-b122bcf45582 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:24.944149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e484502-e39b-4724-b803-282408d42512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:24.944790 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e484502-e39b-4724-b803-282408d42512 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:24.945069 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e484502-e39b-4724-b803-282408d42512 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:24.945347 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e484502-e39b-4724-b803-282408d42512 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:24.945928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e484502-e39b-4724-b803-282408d42512 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:24.946788 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84291ee4-4589-4a34-ae94-6119cd1e643d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:24.947324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e484502-e39b-4724-b803-282408d42512 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:24.947433 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84291ee4-4589-4a34-ae94-6119cd1e643d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:24.947694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84291ee4-4589-4a34-ae94-6119cd1e643d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:24.947960 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84291ee4-4589-4a34-ae94-6119cd1e643d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:24.954855 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84291ee4-4589-4a34-ae94-6119cd1e643d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:24.956279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84291ee4-4589-4a34-ae94-6119cd1e643d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-4e484502-e39b-4724-b803-282408d42512 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:24.956658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:24.957707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:24.958663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 496/982] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Sep 9 16:19:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:24.964095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e484502-e39b-4724-b803-282408d42512 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-84291ee4-4589-4a34-ae94-6119cd1e643d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:19:24.971821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:24.972734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:24.973633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:19:24.973633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:19:24.973633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:19:24.973633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:19:24.973633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:19:24.973633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:19:24.973633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:24.973633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:24.973633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 487/983] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:24.980646 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84291ee4-4589-4a34-ae94-6119cd1e643d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:25.070396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e484502-e39b-4724-b803-282408d42512 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:25.082042 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84291ee4-4589-4a34-ae94-6119cd1e643d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:25.103863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:25.111777 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6be6b7-bd22-4b60-a6bc-105839e30067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:25.112797 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6be6b7-bd22-4b60-a6bc-105839e30067 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:25.113223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6be6b7-bd22-4b60-a6bc-105839e30067 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:25.113490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6be6b7-bd22-4b60-a6bc-105839e30067 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:25.114150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6be6b7-bd22-4b60-a6bc-105839e30067 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:25.114518 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6be6b7-bd22-4b60-a6bc-105839e30067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:25.115971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6be6b7-bd22-4b60-a6bc-105839e30067 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:25.120814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e2de9-2ef1-4363-ac53-6d6878ecf05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:25.121572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 497/984] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Sep 9 16:19:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:25.121681 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e2de9-2ef1-4363-ac53-6d6878ecf05a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:25.121816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e2de9-2ef1-4363-ac53-6d6878ecf05a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:25.122203 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e2de9-2ef1-4363-ac53-6d6878ecf05a None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:25.123014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e2de9-2ef1-4363-ac53-6d6878ecf05a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:25.123389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e2de9-2ef1-4363-ac53-6d6878ecf05a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:25.124963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e2de9-2ef1-4363-ac53-6d6878ecf05a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:25.127381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6be6b7-bd22-4b60-a6bc-105839e30067 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:25.139392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 488/985] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:19:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:25.145158 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e2de9-2ef1-4363-ac53-6d6878ecf05a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:25.150026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:25.230055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6be6b7-bd22-4b60-a6bc-105839e30067 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:25.267563 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161bcb2-d5b2-48c1-a88f-2662aecad021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:25.267771 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e2de9-2ef1-4363-ac53-6d6878ecf05a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:25.268149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161bcb2-d5b2-48c1-a88f-2662aecad021 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:25.268431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161bcb2-d5b2-48c1-a88f-2662aecad021 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:25.268701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161bcb2-d5b2-48c1-a88f-2662aecad021 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:25.269337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161bcb2-d5b2-48c1-a88f-2662aecad021 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:25.269641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161bcb2-d5b2-48c1-a88f-2662aecad021 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:25.271093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161bcb2-d5b2-48c1-a88f-2662aecad021 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:25.276395 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 498/986] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Sep 9 16:19:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:25.277569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1bbe4-7836-4d44-9f47-b19356a72039 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:25.277851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1bbe4-7836-4d44-9f47-b19356a72039 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:25.278128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1bbe4-7836-4d44-9f47-b19356a72039 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:25.279720 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 499/987] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:25.282223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1bbe4-7836-4d44-9f47-b19356a72039 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:25.309349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bba2c0-3cf5-4c20-b773-49203ef75c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:25.309955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bba2c0-3cf5-4c20-b773-49203ef75c36 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:25.310231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bba2c0-3cf5-4c20-b773-49203ef75c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:25.310522 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bba2c0-3cf5-4c20-b773-49203ef75c36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:25.311252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bba2c0-3cf5-4c20-b773-49203ef75c36 None admin] RBAC: Adding query filter params (role.id=4d4fab0f77f847fbb500c58e80767d9e, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=048b4abace244e48b6897daec0277e2f) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:25.311559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bba2c0-3cf5-4c20-b773-49203ef75c36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:25.313017 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bba2c0-3cf5-4c20-b773-49203ef75c36 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:25.319339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 489/988] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:19:25 2022] GET /identity/v3/role_assignments?user.id=048b4abace244e48b6897daec0277e2f&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=4d4fab0f77f847fbb500c58e80767d9e => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:25.320758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5add7db-09fa-4812-92bc-67efd9f06e62 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:25.321044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5add7db-09fa-4812-92bc-67efd9f06e62 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:25.321310 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5add7db-09fa-4812-92bc-67efd9f06e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:25.341941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c5add7db-09fa-4812-92bc-67efd9f06e62 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:25.343574 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c5add7db-09fa-4812-92bc-67efd9f06e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:25.346722 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:25.398462 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1bbe4-7836-4d44-9f47-b19356a72039 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:25.411502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:25.418564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 490/989] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:25.446906 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e4846-24c7-4ce0-acac-f8d377c1ab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:25.447631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e4846-24c7-4ce0-acac-f8d377c1ab64 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:25.447907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e4846-24c7-4ce0-acac-f8d377c1ab64 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:25.448173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e4846-24c7-4ce0-acac-f8d377c1ab64 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:25.448886 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e4846-24c7-4ce0-acac-f8d377c1ab64 None admin] RBAC: Adding query filter params (interface=admin, service_id=e59850358b8d4773a90a46017a9fc48e, region_id=RegionOne) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:25.449191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e4846-24c7-4ce0-acac-f8d377c1ab64 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:25.450593 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e4846-24c7-4ce0-acac-f8d377c1ab64 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:25.461506 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 500/990] 10.222.0.23 () {56 vars in 1249 bytes} [Fri Sep 9 16:19:25 2022] GET /identity/v3/endpoints?service_id=e59850358b8d4773a90a46017a9fc48e&interface=admin®ion_id=RegionOne => generated 193 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:25.462730 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5add7db-09fa-4812-92bc-67efd9f06e62 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:25.569604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5add7db-09fa-4812-92bc-67efd9f06e62 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:25.602541 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485fb15-68d5-4e9d-a529-ccb0b3ae3c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:25.603153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485fb15-68d5-4e9d-a529-ccb0b3ae3c87 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:25.603427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485fb15-68d5-4e9d-a529-ccb0b3ae3c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:25.603696 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485fb15-68d5-4e9d-a529-ccb0b3ae3c87 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:25.604288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485fb15-68d5-4e9d-a529-ccb0b3ae3c87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:25.605653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485fb15-68d5-4e9d-a529-ccb0b3ae3c87 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:25.612407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 491/991] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Sep 9 16:19:25 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:19:25.689310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e4846-24c7-4ce0-acac-f8d377c1ab64 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:25.801554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e4846-24c7-4ce0-acac-f8d377c1ab64 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:25.824383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:25.837829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b275a5-507a-49c5-9b07-45f30a3eef21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:25.838488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b275a5-507a-49c5-9b07-45f30a3eef21 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:25.838779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b275a5-507a-49c5-9b07-45f30a3eef21 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:25.839047 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b275a5-507a-49c5-9b07-45f30a3eef21 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:25.839635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b275a5-507a-49c5-9b07-45f30a3eef21 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:25.841009 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b275a5-507a-49c5-9b07-45f30a3eef21 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-90b275a5-507a-49c5-9b07-45f30a3eef21 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:25.850086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:25.851210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:25.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 501/992] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:25.857336 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485fb15-68d5-4e9d-a529-ccb0b3ae3c87 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:25.921051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:25.962436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485fb15-68d5-4e9d-a529-ccb0b3ae3c87 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:25.997124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf677e0-5bd8-4105-82fc-06437fd7bf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:25.997732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf677e0-5bd8-4105-82fc-06437fd7bf6d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:25.998003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf677e0-5bd8-4105-82fc-06437fd7bf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:25.998263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf677e0-5bd8-4105-82fc-06437fd7bf6d None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:25.998913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf677e0-5bd8-4105-82fc-06437fd7bf6d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:25.999207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf677e0-5bd8-4105-82fc-06437fd7bf6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:26.000600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf677e0-5bd8-4105-82fc-06437fd7bf6d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:26.010140 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 492/993] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Sep 9 16:19:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:26.011504 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b275a5-507a-49c5-9b07-45f30a3eef21 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:26.123326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b275a5-507a-49c5-9b07-45f30a3eef21 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:26.157181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713b767-0fda-4def-ad29-99d59a8734ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:26.157792 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713b767-0fda-4def-ad29-99d59a8734ab None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:26.158069 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713b767-0fda-4def-ad29-99d59a8734ab None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:26.158337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713b767-0fda-4def-ad29-99d59a8734ab None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:26.159061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713b767-0fda-4def-ad29-99d59a8734ab None admin] RBAC: Adding query filter params (type=compute) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:26.159370 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713b767-0fda-4def-ad29-99d59a8734ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:26.160733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713b767-0fda-4def-ad29-99d59a8734ab None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:26.170407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 502/994] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Sep 9 16:19:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:26.172010 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf677e0-5bd8-4105-82fc-06437fd7bf6d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:26.278360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf677e0-5bd8-4105-82fc-06437fd7bf6d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:26.304346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:26.314839 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd62c60-8b08-41df-97bd-989025507441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:26.315803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd62c60-8b08-41df-97bd-989025507441 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:26.316077 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd62c60-8b08-41df-97bd-989025507441 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:26.316339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd62c60-8b08-41df-97bd-989025507441 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:26.317035 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd62c60-8b08-41df-97bd-989025507441 None admin] RBAC: Adding query filter params (interface=public, service_id=a9566b0d654a4f3ebe395f157145a6e7, region_id=RegionOne) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:26.317332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd62c60-8b08-41df-97bd-989025507441 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:26.318742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd62c60-8b08-41df-97bd-989025507441 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:26.328667 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 493/995] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Sep 9 16:19:26 2022] GET /identity/v3/endpoints?service_id=a9566b0d654a4f3ebe395f157145a6e7&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:26.330145 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713b767-0fda-4def-ad29-99d59a8734ab None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:26.381200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:26.431491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713b767-0fda-4def-ad29-99d59a8734ab None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:26.464675 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d083180-9fc5-4a5c-a6f1-c292bde1c4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:26.465285 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d083180-9fc5-4a5c-a6f1-c292bde1c4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:26.465565 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d083180-9fc5-4a5c-a6f1-c292bde1c4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:26.465839 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d083180-9fc5-4a5c-a6f1-c292bde1c4c1 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:26.466447 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d083180-9fc5-4a5c-a6f1-c292bde1c4c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:26.467849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d083180-9fc5-4a5c-a6f1-c292bde1c4c1 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:26.474473 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 503/996] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Sep 9 16:19:26 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:19:27.513888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93267c6a-3691-4da6-9d28-25975e0aa964 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:27.514424 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93267c6a-3691-4da6-9d28-25975e0aa964 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:27.514424 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93267c6a-3691-4da6-9d28-25975e0aa964 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:27.515870 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 494/997] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:27.521227 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b60499-456b-4a52-9771-9f2308ba61df None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:27.521641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b60499-456b-4a52-9771-9f2308ba61df None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:27.521741 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b60499-456b-4a52-9771-9f2308ba61df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:27.540712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d9b60499-456b-4a52-9771-9f2308ba61df None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:27.541892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d9b60499-456b-4a52-9771-9f2308ba61df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:27.610709 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 504/998] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:19:27.767571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41dd47-c44e-4164-868c-83dfbe16879d None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:27.767785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41dd47-c44e-4164-868c-83dfbe16879d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:27.768052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41dd47-c44e-4164-868c-83dfbe16879d None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:27.769517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 495/999] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:27.774742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bedf6c-47fd-463f-a317-41e5853ed426 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:27.775031 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bedf6c-47fd-463f-a317-41e5853ed426 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:27.775267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bedf6c-47fd-463f-a317-41e5853ed426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:27.793712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-17bedf6c-47fd-463f-a317-41e5853ed426 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:27.794893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-17bedf6c-47fd-463f-a317-41e5853ed426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:27.835309 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:27.862261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 505/1000] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:27.875411 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41dd47-c44e-4164-868c-83dfbe16879d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:27.976307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41dd47-c44e-4164-868c-83dfbe16879d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:28.010263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58a5f4-23cc-4031-9a8d-ff680c890035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:28.010917 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58a5f4-23cc-4031-9a8d-ff680c890035 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:28.011188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58a5f4-23cc-4031-9a8d-ff680c890035 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:28.011455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58a5f4-23cc-4031-9a8d-ff680c890035 None admin] PATH_INFO: `/v3/services/image` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:28.012028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58a5f4-23cc-4031-9a8d-ff680c890035 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:28.013390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58a5f4-23cc-4031-9a8d-ff680c890035 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-5b58a5f4-23cc-4031-9a8d-ff680c890035 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:28.022183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:28.026787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:28.027663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 496/1001] 10.222.0.23 () {56 vars in 1105 bytes} [Fri Sep 9 16:19:27 2022] GET /identity/v3/services/image => generated 86 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:28.029967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bedf6c-47fd-463f-a317-41e5853ed426 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:28.124832 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58a5f4-23cc-4031-9a8d-ff680c890035 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:28.141488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bedf6c-47fd-463f-a317-41e5853ed426 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:28.178390 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c56806-7ba2-4652-b338-8f46f72211ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:28.179024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c56806-7ba2-4652-b338-8f46f72211ff None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:28.179300 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c56806-7ba2-4652-b338-8f46f72211ff None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:28.179566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c56806-7ba2-4652-b338-8f46f72211ff None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:28.180301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c56806-7ba2-4652-b338-8f46f72211ff None admin] RBAC: Adding query filter params (name=image) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:28.180602 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c56806-7ba2-4652-b338-8f46f72211ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:28.181963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c56806-7ba2-4652-b338-8f46f72211ff None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:28.183734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:28.193953 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 506/1002] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:28.195018 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c56806-7ba2-4652-b338-8f46f72211ff None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:28.230032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58a5f4-23cc-4031-9a8d-ff680c890035 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:28.235909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:28.269387 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96866265-1403-4e30-b36f-e5e571ced6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:28.270072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96866265-1403-4e30-b36f-e5e571ced6ce None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:28.270356 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96866265-1403-4e30-b36f-e5e571ced6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:28.270750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96866265-1403-4e30-b36f-e5e571ced6ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:28.271341 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96866265-1403-4e30-b36f-e5e571ced6ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:28.272785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96866265-1403-4e30-b36f-e5e571ced6ce None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-96866265-1403-4e30-b36f-e5e571ced6ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:28.284265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:28.285342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:28.287640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 497/1003] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Sep 9 16:19:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:28.291493 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96866265-1403-4e30-b36f-e5e571ced6ce None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:28.304217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c56806-7ba2-4652-b338-8f46f72211ff None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:28.344369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530abdd-5940-44e5-b293-8bc8fb8f59a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:28.344977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530abdd-5940-44e5-b293-8bc8fb8f59a9 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:28.345306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530abdd-5940-44e5-b293-8bc8fb8f59a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:28.345538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530abdd-5940-44e5-b293-8bc8fb8f59a9 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:28.346642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530abdd-5940-44e5-b293-8bc8fb8f59a9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:28.347151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530abdd-5940-44e5-b293-8bc8fb8f59a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:28.348614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530abdd-5940-44e5-b293-8bc8fb8f59a9 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:28.362533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 507/1004] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:28 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:28.402727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96866265-1403-4e30-b36f-e5e571ced6ce None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:28.422115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:28.442541 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36824d44-47a1-4e2a-aef3-bbe622f588fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:28.443171 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36824d44-47a1-4e2a-aef3-bbe622f588fa None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:28.443481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36824d44-47a1-4e2a-aef3-bbe622f588fa None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:28.443758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36824d44-47a1-4e2a-aef3-bbe622f588fa None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:28.444402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36824d44-47a1-4e2a-aef3-bbe622f588fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:28.444782 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36824d44-47a1-4e2a-aef3-bbe622f588fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:28.446222 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36824d44-47a1-4e2a-aef3-bbe622f588fa None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:28.456501 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 498/1005] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Sep 9 16:19:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:28.458312 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530abdd-5940-44e5-b293-8bc8fb8f59a9 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:28.571415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530abdd-5940-44e5-b293-8bc8fb8f59a9 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:28.573019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:28.606960 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954e800-788d-422e-a4bc-5fcb17d3c2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:28.607573 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954e800-788d-422e-a4bc-5fcb17d3c2bb None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:28.607848 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954e800-788d-422e-a4bc-5fcb17d3c2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:28.608114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954e800-788d-422e-a4bc-5fcb17d3c2bb None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:28.608738 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954e800-788d-422e-a4bc-5fcb17d3c2bb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:28.609036 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954e800-788d-422e-a4bc-5fcb17d3c2bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:28.610396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954e800-788d-422e-a4bc-5fcb17d3c2bb None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:28.615575 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 508/1006] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Sep 9 16:19:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:28.621191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36824d44-47a1-4e2a-aef3-bbe622f588fa None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:28.621686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf5497-ad67-4371-ae33-ae7158c7bd88 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:28.621777 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf5497-ad67-4371-ae33-ae7158c7bd88 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:28.622030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf5497-ad67-4371-ae33-ae7158c7bd88 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:28.623513 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 509/1007] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:28.628306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681d077-a2e1-40c2-8c01-8767a1c3ca6e None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:28.628590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681d077-a2e1-40c2-8c01-8767a1c3ca6e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:28.628864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681d077-a2e1-40c2-8c01-8767a1c3ca6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:28.647235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f681d077-a2e1-40c2-8c01-8767a1c3ca6e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:28.648429 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f681d077-a2e1-40c2-8c01-8767a1c3ca6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:28.719814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 510/1008] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:28 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:19:28.730435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36824d44-47a1-4e2a-aef3-bbe622f588fa None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:28.764810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ad6ca-6725-441f-95cc-2ceee2d6363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:28.765669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ad6ca-6725-441f-95cc-2ceee2d6363c None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:28.766090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ad6ca-6725-441f-95cc-2ceee2d6363c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:28.766640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ad6ca-6725-441f-95cc-2ceee2d6363c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:28.767493 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ad6ca-6725-441f-95cc-2ceee2d6363c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:28.769003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ad6ca-6725-441f-95cc-2ceee2d6363c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:28.787746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 499/1009] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:19:28 2022] POST /identity/v3/endpoints => generated 325 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:28.981678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681d077-a2e1-40c2-8c01-8767a1c3ca6e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:29.078839 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:29.085107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681d077-a2e1-40c2-8c01-8767a1c3ca6e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:29.127029 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454a478-961b-4843-971b-7223756c52e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:29.127676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454a478-961b-4843-971b-7223756c52e1 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:29.127950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454a478-961b-4843-971b-7223756c52e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:29.128215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454a478-961b-4843-971b-7223756c52e1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:29.128800 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454a478-961b-4843-971b-7223756c52e1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:29.130524 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454a478-961b-4843-971b-7223756c52e1 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-8454a478-961b-4843-971b-7223756c52e1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:29.139541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:29.140545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:29.141369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 511/1010] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:29.148358 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ad6ca-6725-441f-95cc-2ceee2d6363c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:29.194395 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:29.257312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ad6ca-6725-441f-95cc-2ceee2d6363c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:29.292371 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d14d9-6467-4463-af08-aaf867aa1bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:29.292985 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d14d9-6467-4463-af08-aaf867aa1bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:29.293258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d14d9-6467-4463-af08-aaf867aa1bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:29.293523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d14d9-6467-4463-af08-aaf867aa1bf9 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:29.294208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d14d9-6467-4463-af08-aaf867aa1bf9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:29.294531 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d14d9-6467-4463-af08-aaf867aa1bf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:29.295907 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d14d9-6467-4463-af08-aaf867aa1bf9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:29.300765 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 500/1011] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Sep 9 16:19:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:29.306419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454a478-961b-4843-971b-7223756c52e1 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:29.405528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454a478-961b-4843-971b-7223756c52e1 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:29.439018 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb73037-85c5-4e51-8dbc-0c38562f970b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:29.439632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb73037-85c5-4e51-8dbc-0c38562f970b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:29.440272 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb73037-85c5-4e51-8dbc-0c38562f970b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:29.440542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb73037-85c5-4e51-8dbc-0c38562f970b None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:29.441167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb73037-85c5-4e51-8dbc-0c38562f970b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:29.441468 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb73037-85c5-4e51-8dbc-0c38562f970b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:29.442920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb73037-85c5-4e51-8dbc-0c38562f970b None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:29.447883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 512/1012] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Sep 9 16:19:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:29.453471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d14d9-6467-4463-af08-aaf867aa1bf9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:29.552105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d14d9-6467-4463-af08-aaf867aa1bf9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:29.559244 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:29.592693 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357631c-22eb-46db-a391-ff9de475a3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:29.593305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357631c-22eb-46db-a391-ff9de475a3e9 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:29.593581 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357631c-22eb-46db-a391-ff9de475a3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:29.593848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357631c-22eb-46db-a391-ff9de475a3e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:29.594654 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357631c-22eb-46db-a391-ff9de475a3e9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:29.595942 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357631c-22eb-46db-a391-ff9de475a3e9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:29.616182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 501/1013] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:19:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:30.524538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0902f-5983-4af0-b608-5bef9382d93e None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:30.525098 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0902f-5983-4af0-b608-5bef9382d93e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:30.525098 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0902f-5983-4af0-b608-5bef9382d93e None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:30.526607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 513/1014] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:30.531722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c61e05-2970-46b6-8e48-7193f6174f83 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:30.532139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c61e05-2970-46b6-8e48-7193f6174f83 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:30.532328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c61e05-2970-46b6-8e48-7193f6174f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:30.551202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-98c61e05-2970-46b6-8e48-7193f6174f83 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:30.552398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-98c61e05-2970-46b6-8e48-7193f6174f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:30.630405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 502/1015] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:30 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:30.901763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0902f-5983-4af0-b608-5bef9382d93e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:30.925924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:30.926112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6266acf-64e0-41d9-bb23-fbeafe5fc4ad None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:30.926342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6266acf-64e0-41d9-bb23-fbeafe5fc4ad None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:30.926561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6266acf-64e0-41d9-bb23-fbeafe5fc4ad None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:30.928261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 503/1016] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:30.933416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75c89d-5aa2-4d32-8af6-907aee0fbd17 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:30.933693 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75c89d-5aa2-4d32-8af6-907aee0fbd17 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:30.933959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75c89d-5aa2-4d32-8af6-907aee0fbd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:30.952380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ce75c89d-5aa2-4d32-8af6-907aee0fbd17 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:30.953553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ce75c89d-5aa2-4d32-8af6-907aee0fbd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:31.017900 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0902f-5983-4af0-b608-5bef9382d93e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:31.022990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 504/1017] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:30 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:19:31.058587 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeba610-2d1c-4e74-9594-d22599852558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:31.059583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba610-2d1c-4e74-9594-d22599852558 None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:31.060055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba610-2d1c-4e74-9594-d22599852558 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:31.060514 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba610-2d1c-4e74-9594-d22599852558 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:31.061273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeba610-2d1c-4e74-9594-d22599852558 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:31.062705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeba610-2d1c-4e74-9594-d22599852558 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:31.081505 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 514/1018] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Sep 9 16:19:30 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:31.284963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75c89d-5aa2-4d32-8af6-907aee0fbd17 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:31.303832 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:31.386231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75c89d-5aa2-4d32-8af6-907aee0fbd17 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:31.429230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a085d-04be-428d-bba0-4c5c65c368e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:31.429862 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a085d-04be-428d-bba0-4c5c65c368e9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:31.430131 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a085d-04be-428d-bba0-4c5c65c368e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:31.430392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a085d-04be-428d-bba0-4c5c65c368e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:31.431536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a085d-04be-428d-bba0-4c5c65c368e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:31.432958 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a085d-04be-428d-bba0-4c5c65c368e9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:31.434702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 505/1019] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:31.440824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeba610-2d1c-4e74-9594-d22599852558 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:31.522777 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:31.545264 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeba610-2d1c-4e74-9594-d22599852558 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:31.579639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde29cb-a7d0-42fa-b5c1-eab7ba6e76dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:31.580246 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde29cb-a7d0-42fa-b5c1-eab7ba6e76dd None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:31.580515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde29cb-a7d0-42fa-b5c1-eab7ba6e76dd None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:31.580778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde29cb-a7d0-42fa-b5c1-eab7ba6e76dd None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:31.581429 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde29cb-a7d0-42fa-b5c1-eab7ba6e76dd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:31.583067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde29cb-a7d0-42fa-b5c1-eab7ba6e76dd None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:31.621124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 515/1020] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Sep 9 16:19:31 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:31.878335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a20658-3a75-4be5-bac7-d9ba0a515786 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:31.878930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a20658-3a75-4be5-bac7-d9ba0a515786 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:31.878930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a20658-3a75-4be5-bac7-d9ba0a515786 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:31.880482 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 506/1021] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:31.885513 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f96ffc-c687-4903-804d-1e071e8cd349 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:31.885797 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f96ffc-c687-4903-804d-1e071e8cd349 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:31.886065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f96ffc-c687-4903-804d-1e071e8cd349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:31.904095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-b1f96ffc-c687-4903-804d-1e071e8cd349 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:31.905267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b1f96ffc-c687-4903-804d-1e071e8cd349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:31.974534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 516/1022] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:32.237222 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a20658-3a75-4be5-bac7-d9ba0a515786 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:32.327680 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:32.347854 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a20658-3a75-4be5-bac7-d9ba0a515786 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:32.383515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6fdb9-bc40-4e23-bd1d-f9fc6bea2814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:32.384154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6fdb9-bc40-4e23-bd1d-f9fc6bea2814 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:32.384430 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6fdb9-bc40-4e23-bd1d-f9fc6bea2814 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:32.384698 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6fdb9-bc40-4e23-bd1d-f9fc6bea2814 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:32.396266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6fdb9-bc40-4e23-bd1d-f9fc6bea2814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-abf6fdb9-bc40-4e23-bd1d-f9fc6bea2814 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:19:32.401566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:32.402523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:32.403703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:32.403703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:32.403703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:32.403703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:32.403703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:19:32.403703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:19:32.403703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:19:32.403703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:19:32.403703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:19:32.403703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:19:32.403703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 16:19:32.403703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:32.403703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 507/1023] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Sep 9 16:19:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:32.409838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f96ffc-c687-4903-804d-1e071e8cd349 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:32.473658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:32.513453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f96ffc-c687-4903-804d-1e071e8cd349 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:32.549711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426693f6-cbad-412d-ab20-7a1a62edddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:32.550416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426693f6-cbad-412d-ab20-7a1a62edddcd None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:32.550661 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426693f6-cbad-412d-ab20-7a1a62edddcd None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:32.550905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426693f6-cbad-412d-ab20-7a1a62edddcd None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:32.551571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426693f6-cbad-412d-ab20-7a1a62edddcd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:32.551925 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426693f6-cbad-412d-ab20-7a1a62edddcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:32.553296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426693f6-cbad-412d-ab20-7a1a62edddcd None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:32.561855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 517/1024] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Sep 9 16:19:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:32.568227 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6fdb9-bc40-4e23-bd1d-f9fc6bea2814 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:32.665798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6fdb9-bc40-4e23-bd1d-f9fc6bea2814 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:32.699925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf2723-c0d4-4f96-8e8e-2317fa26c912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:32.700566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf2723-c0d4-4f96-8e8e-2317fa26c912 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:32.700824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf2723-c0d4-4f96-8e8e-2317fa26c912 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:32.701078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf2723-c0d4-4f96-8e8e-2317fa26c912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:32.711517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf2723-c0d4-4f96-8e8e-2317fa26c912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:32.712949 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf2723-c0d4-4f96-8e8e-2317fa26c912 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:32.714671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-28bf2723-c0d4-4f96-8e8e-2317fa26c912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:32.729112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:32.730065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:32.730980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:32.730980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:32.730980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:32.735098 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 508/1025] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:32.736530 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426693f6-cbad-412d-ab20-7a1a62edddcd None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:32.838951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426693f6-cbad-412d-ab20-7a1a62edddcd None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:32.861835 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:32.874482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7acf57-6b25-4202-bd0a-84afc9c315d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:32.875122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7acf57-6b25-4202-bd0a-84afc9c315d1 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:32.875398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7acf57-6b25-4202-bd0a-84afc9c315d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:32.875668 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7acf57-6b25-4202-bd0a-84afc9c315d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:32.876316 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7acf57-6b25-4202-bd0a-84afc9c315d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:32.876614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7acf57-6b25-4202-bd0a-84afc9c315d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:32.877986 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7acf57-6b25-4202-bd0a-84afc9c315d1 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:32.890416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 518/1026] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:32.896061 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf2723-c0d4-4f96-8e8e-2317fa26c912 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:32.996263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf2723-c0d4-4f96-8e8e-2317fa26c912 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:33.029447 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f2980-bcb5-46d9-acee-181a4a30bcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:33.030081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f2980-bcb5-46d9-acee-181a4a30bcbc None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:33.030356 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f2980-bcb5-46d9-acee-181a4a30bcbc None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:33.030655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f2980-bcb5-46d9-acee-181a4a30bcbc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:33.038121 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f2980-bcb5-46d9-acee-181a4a30bcbc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:33.039596 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f2980-bcb5-46d9-acee-181a4a30bcbc None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-f83f2980-bcb5-46d9-acee-181a4a30bcbc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:19:33.050530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:33.051434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:33.052454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:33.052454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:33.052454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:19:33.052454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:19:33.052454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:19:33.052454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:19:33.052454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:19:33.052454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:19:33.052454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:19:33.052454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:33.054855 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 509/1027] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Sep 9 16:19:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:33.057445 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7acf57-6b25-4202-bd0a-84afc9c315d1 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:33.162280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7acf57-6b25-4202-bd0a-84afc9c315d1 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:33.201174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b41864-bdf7-4781-b74f-a1a74e72045c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:33.201926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b41864-bdf7-4781-b74f-a1a74e72045c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:33.202204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b41864-bdf7-4781-b74f-a1a74e72045c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:33.202467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b41864-bdf7-4781-b74f-a1a74e72045c None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:33.203241 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b41864-bdf7-4781-b74f-a1a74e72045c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:33.203543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b41864-bdf7-4781-b74f-a1a74e72045c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:33.204971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b41864-bdf7-4781-b74f-a1a74e72045c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:33.222489 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 519/1028] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Sep 9 16:19:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:33.228332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f2980-bcb5-46d9-acee-181a4a30bcbc None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:33.284086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:33.288059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cccb7-70bc-4c5c-a712-7440bb2aede2 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:33.288338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cccb7-70bc-4c5c-a712-7440bb2aede2 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:33.288607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cccb7-70bc-4c5c-a712-7440bb2aede2 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:33.289945 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 520/1029] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:33.295475 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b643ad0-d543-4a84-bb1d-4343adf618c3 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:33.295750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b643ad0-d543-4a84-bb1d-4343adf618c3 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:33.296019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b643ad0-d543-4a84-bb1d-4343adf618c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:33.315670 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-6b643ad0-d543-4a84-bb1d-4343adf618c3 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:33.316849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-6b643ad0-d543-4a84-bb1d-4343adf618c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:33.341325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f2980-bcb5-46d9-acee-181a4a30bcbc None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:33.353801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:33.379455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beed540-225e-45d3-b2fe-6ad32a06e3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:33.380117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beed540-225e-45d3-b2fe-6ad32a06e3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:33.380400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beed540-225e-45d3-b2fe-6ad32a06e3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:33.380672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beed540-225e-45d3-b2fe-6ad32a06e3f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:33.388404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 521/1030] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:19:33.451765 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beed540-225e-45d3-b2fe-6ad32a06e3f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:33.453142 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beed540-225e-45d3-b2fe-6ad32a06e3f7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:33.508045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:33.554574 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 510/1031] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:19:33 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:19:33.560306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b643ad0-d543-4a84-bb1d-4343adf618c3 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:33.651174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beed540-225e-45d3-b2fe-6ad32a06e3f7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:33.660862 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b643ad0-d543-4a84-bb1d-4343adf618c3 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:33.704815 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312c457-fc2a-411f-81f5-2027a2767ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:33.705372 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312c457-fc2a-411f-81f5-2027a2767ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:33.705658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312c457-fc2a-411f-81f5-2027a2767ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:33.705923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312c457-fc2a-411f-81f5-2027a2767ae0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:33.716714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312c457-fc2a-411f-81f5-2027a2767ae0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:33.718168 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312c457-fc2a-411f-81f5-2027a2767ae0 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-9312c457-fc2a-411f-81f5-2027a2767ae0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:33.735467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:33.736455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:33.742446 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:33.742446 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:33.742446 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:33.742446 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 522/1032] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:33.742897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312c457-fc2a-411f-81f5-2027a2767ae0 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:33.758318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beed540-225e-45d3-b2fe-6ad32a06e3f7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:33.770421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:33.794943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d1e54-85a1-4942-ba9b-4d37b6c714fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:33.795573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d1e54-85a1-4942-ba9b-4d37b6c714fa None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:33.795853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d1e54-85a1-4942-ba9b-4d37b6c714fa None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:33.796117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d1e54-85a1-4942-ba9b-4d37b6c714fa None admin] PATH_INFO: `/v3/services/image` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:33.796699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d1e54-85a1-4942-ba9b-4d37b6c714fa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:33.798066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d1e54-85a1-4942-ba9b-4d37b6c714fa None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-cc5d1e54-85a1-4942-ba9b-4d37b6c714fa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:33.807248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:33.808204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:33.809052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 511/1033] 10.222.0.23 () {56 vars in 1105 bytes} [Fri Sep 9 16:19:33 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:33.810215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b972a-9efc-4c5b-afb3-90c5650472a6 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:33.810215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b972a-9efc-4c5b-afb3-90c5650472a6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:33.810215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b972a-9efc-4c5b-afb3-90c5650472a6 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:33.811229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 512/1034] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:33.814049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b972a-9efc-4c5b-afb3-90c5650472a6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:33.833219 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:33.849061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312c457-fc2a-411f-81f5-2027a2767ae0 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:33.881181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1711fa-3891-4323-bf41-03c774effacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:33.881794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1711fa-3891-4323-bf41-03c774effacb None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:33.882073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1711fa-3891-4323-bf41-03c774effacb None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:33.882339 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1711fa-3891-4323-bf41-03c774effacb None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:33.883008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1711fa-3891-4323-bf41-03c774effacb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:33.883312 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1711fa-3891-4323-bf41-03c774effacb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:33.884708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1711fa-3891-4323-bf41-03c774effacb None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:33.896113 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 523/1035] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:33.897262 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118fb7b-448f-430e-867b-a575d05b2c74 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:33.897542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118fb7b-448f-430e-867b-a575d05b2c74 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:33.897824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118fb7b-448f-430e-867b-a575d05b2c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:33.914375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b972a-9efc-4c5b-afb3-90c5650472a6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:33.915042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-0118fb7b-448f-430e-867b-a575d05b2c74 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:33.916223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0118fb7b-448f-430e-867b-a575d05b2c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:33.948484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f5f35-b2f0-4283-9a55-ad67bbca2d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:33.949202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f5f35-b2f0-4283-9a55-ad67bbca2d3d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:33.949482 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f5f35-b2f0-4283-9a55-ad67bbca2d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:33.949753 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f5f35-b2f0-4283-9a55-ad67bbca2d3d None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:33.950379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f5f35-b2f0-4283-9a55-ad67bbca2d3d None admin] RBAC: Adding query filter params (name=image) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:33.950710 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f5f35-b2f0-4283-9a55-ad67bbca2d3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:33.952073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f5f35-b2f0-4283-9a55-ad67bbca2d3d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:33.957005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 513/1036] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:33.957745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f5f35-b2f0-4283-9a55-ad67bbca2d3d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:33.985226 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 524/1037] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:33.985895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118fb7b-448f-430e-867b-a575d05b2c74 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:34.000700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:34.059120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:34.081847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f5f35-b2f0-4283-9a55-ad67bbca2d3d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:34.096190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118fb7b-448f-430e-867b-a575d05b2c74 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:34.120129 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a0b2c-acea-42af-8064-76134038c888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:34.121062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0b2c-acea-42af-8064-76134038c888 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:34.121338 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0b2c-acea-42af-8064-76134038c888 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:34.121606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0b2c-acea-42af-8064-76134038c888 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:34.128438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a0b2c-acea-42af-8064-76134038c888 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:34.129974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a0b2c-acea-42af-8064-76134038c888 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:34.131120 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a735f-08f4-4bb7-a371-4a1f1c2e00fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:34.131736 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a735f-08f4-4bb7-a371-4a1f1c2e00fd None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:34.132017 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a735f-08f4-4bb7-a371-4a1f1c2e00fd None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:34.132289 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a735f-08f4-4bb7-a371-4a1f1c2e00fd None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:34.132927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a735f-08f4-4bb7-a371-4a1f1c2e00fd None admin] RBAC: Adding query filter params (type=image) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:34.133237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a735f-08f4-4bb7-a371-4a1f1c2e00fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:34.134632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a735f-08f4-4bb7-a371-4a1f1c2e00fd None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-8f6a0b2c-acea-42af-8064-76134038c888 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:19:34.141536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:34.142405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:34.143801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:19:34.143801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:19:34.143801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:19:34.143801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:19:34.143801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:19:34.143801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:19:34.143801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:34.143801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:34.144941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 525/1038] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:34.145841 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a735f-08f4-4bb7-a371-4a1f1c2e00fd None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:34.146815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 514/1039] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:34.148206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a0b2c-acea-42af-8064-76134038c888 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:34.225356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:34.247954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a735f-08f4-4bb7-a371-4a1f1c2e00fd None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:34.254172 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a0b2c-acea-42af-8064-76134038c888 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:34.281067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45679f5a-dd2c-4afd-bce1-b3955bc4c111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:34.281686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45679f5a-dd2c-4afd-bce1-b3955bc4c111 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:34.281961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45679f5a-dd2c-4afd-bce1-b3955bc4c111 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:34.282234 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45679f5a-dd2c-4afd-bce1-b3955bc4c111 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:34.283041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45679f5a-dd2c-4afd-bce1-b3955bc4c111 None admin] RBAC: Adding query filter params (interface=public, service_id=3ca33c25c05a43df9c9997be088e484e, region_id=RegionOne) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:34.283347 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45679f5a-dd2c-4afd-bce1-b3955bc4c111 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:34.284715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45679f5a-dd2c-4afd-bce1-b3955bc4c111 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:34.287084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e251868-2a26-461d-967d-551655b94d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:34.287745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e251868-2a26-461d-967d-551655b94d9e None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:34.288026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e251868-2a26-461d-967d-551655b94d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:34.288284 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e251868-2a26-461d-967d-551655b94d9e None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:34.288961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e251868-2a26-461d-967d-551655b94d9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:34.289260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e251868-2a26-461d-967d-551655b94d9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:34.289709 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 526/1040] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Sep 9 16:19:34 2022] GET /identity/v3/endpoints?service_id=3ca33c25c05a43df9c9997be088e484e&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:34.290492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45679f5a-dd2c-4afd-bce1-b3955bc4c111 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:34.290762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e251868-2a26-461d-967d-551655b94d9e None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:34.303299 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 515/1041] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:19:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:34.304070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e251868-2a26-461d-967d-551655b94d9e None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:34.306715 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:34.392690 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45679f5a-dd2c-4afd-bce1-b3955bc4c111 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:34.400849 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e251868-2a26-461d-967d-551655b94d9e None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:34.425975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac1a6b-f7aa-4819-a7c3-510ed2fdc3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:34.426717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac1a6b-f7aa-4819-a7c3-510ed2fdc3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:34.426999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac1a6b-f7aa-4819-a7c3-510ed2fdc3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:34.427267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac1a6b-f7aa-4819-a7c3-510ed2fdc3d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:34.428390 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac1a6b-f7aa-4819-a7c3-510ed2fdc3d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:34.429824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac1a6b-f7aa-4819-a7c3-510ed2fdc3d4 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:34.431427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 527/1042] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:34.432181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac1a6b-f7aa-4819-a7c3-510ed2fdc3d4 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:34.433267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbefa00-0a44-4092-9e0c-b0bf40042302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:34.433876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbefa00-0a44-4092-9e0c-b0bf40042302 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:34.434156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbefa00-0a44-4092-9e0c-b0bf40042302 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:34.434468 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbefa00-0a44-4092-9e0c-b0bf40042302 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:34.435105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbefa00-0a44-4092-9e0c-b0bf40042302 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:34.436720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbefa00-0a44-4092-9e0c-b0bf40042302 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:34.445184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 516/1043] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Sep 9 16:19:34 2022] GET /identity/v3/services => generated 1957 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:19:34.445184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbefa00-0a44-4092-9e0c-b0bf40042302 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:34.446698 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:34.528858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:34.542074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac1a6b-f7aa-4819-a7c3-510ed2fdc3d4 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:34.570051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbefa00-0a44-4092-9e0c-b0bf40042302 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:34.578073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28ad94-f11e-40b1-9421-0e245448ae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:34.578727 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28ad94-f11e-40b1-9421-0e245448ae84 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:34.579019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28ad94-f11e-40b1-9421-0e245448ae84 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:34.579286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28ad94-f11e-40b1-9421-0e245448ae84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:34.580074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28ad94-f11e-40b1-9421-0e245448ae84 None admin] RBAC: Adding query filter params (role.id=4d4fab0f77f847fbb500c58e80767d9e, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=152a41ad598b4b01b1881d928c53c848) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:34.580344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28ad94-f11e-40b1-9421-0e245448ae84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:34.581822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28ad94-f11e-40b1-9421-0e245448ae84 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:34.598857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 528/1044] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:19:34 2022] GET /identity/v3/role_assignments?user.id=152a41ad598b4b01b1881d928c53c848&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=4d4fab0f77f847fbb500c58e80767d9e => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:34.620843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1dd16-a730-4cc6-ac4f-a5212172431c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:34.621561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1dd16-a730-4cc6-ac4f-a5212172431c None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:34.621897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1dd16-a730-4cc6-ac4f-a5212172431c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:34.622803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1dd16-a730-4cc6-ac4f-a5212172431c None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:34.623552 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1dd16-a730-4cc6-ac4f-a5212172431c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:34.625063 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1dd16-a730-4cc6-ac4f-a5212172431c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:34.659277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 517/1045] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:19:34 2022] POST /identity/v3/users => generated 285 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:36.605986 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c445f5a-cb60-4a8f-a814-9b696e813f64 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:36.606568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c445f5a-cb60-4a8f-a814-9b696e813f64 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:36.606568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c445f5a-cb60-4a8f-a814-9b696e813f64 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:36.608016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 529/1046] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:36.614295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe076e-2789-4661-9be8-03703c9b168e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:36.614940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe076e-2789-4661-9be8-03703c9b168e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:36.614940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe076e-2789-4661-9be8-03703c9b168e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:36.634932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-14fe076e-2789-4661-9be8-03703c9b168e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:36.636159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-14fe076e-2789-4661-9be8-03703c9b168e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:36.706986 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 518/1047] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:36.729335 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a393824-4a5c-4dd4-bc35-26aa1d4e99d3 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:36.729557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a393824-4a5c-4dd4-bc35-26aa1d4e99d3 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:36.729844 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a393824-4a5c-4dd4-bc35-26aa1d4e99d3 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:36.731508 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 530/1048] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:36.737266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339ecc0-20d3-4da3-ab6e-b45c560cd3ba None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:36.737516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339ecc0-20d3-4da3-ab6e-b45c560cd3ba None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:36.737815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339ecc0-20d3-4da3-ab6e-b45c560cd3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:36.757633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3339ecc0-20d3-4da3-ab6e-b45c560cd3ba None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:36.758858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3339ecc0-20d3-4da3-ab6e-b45c560cd3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:36.806808 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:36.829146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 519/1049] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:36.839008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd3339-3afc-4f7e-8be0-87c979e96db7 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:36.839290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd3339-3afc-4f7e-8be0-87c979e96db7 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:36.839557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd3339-3afc-4f7e-8be0-87c979e96db7 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:36.841054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 531/1050] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:36.846172 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d5a52-704e-4245-be37-e07918632aee None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:36.846414 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d5a52-704e-4245-be37-e07918632aee None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:36.846755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d5a52-704e-4245-be37-e07918632aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:36.865163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0d2d5a52-704e-4245-be37-e07918632aee None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:36.866248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0d2d5a52-704e-4245-be37-e07918632aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:36.932711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 520/1051] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:19:36.967317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd3339-3afc-4f7e-8be0-87c979e96db7 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:37.043430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:37.071541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd3339-3afc-4f7e-8be0-87c979e96db7 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:37.093137 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d5a52-704e-4245-be37-e07918632aee None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:37.114105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae48a0-62b5-4963-b3ee-29c15312ae1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:37.114789 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae48a0-62b5-4963-b3ee-29c15312ae1c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:37.115072 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae48a0-62b5-4963-b3ee-29c15312ae1c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:37.115402 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae48a0-62b5-4963-b3ee-29c15312ae1c None admin] PATH_INFO: `/v3/services/image` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:37.115930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae48a0-62b5-4963-b3ee-29c15312ae1c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:37.117310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae48a0-62b5-4963-b3ee-29c15312ae1c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-00ae48a0-62b5-4963-b3ee-29c15312ae1c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:37.126611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:37.127603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:37.128471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 532/1052] 10.222.0.23 () {56 vars in 1105 bytes} [Fri Sep 9 16:19:36 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:37.133847 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae48a0-62b5-4963-b3ee-29c15312ae1c None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:37.204299 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d5a52-704e-4245-be37-e07918632aee None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:37.217036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:37.233139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae48a0-62b5-4963-b3ee-29c15312ae1c None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:37.238125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f1f11-0c7f-4bcd-ac5c-1d6a8017ade4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:37.238782 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f1f11-0c7f-4bcd-ac5c-1d6a8017ade4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:37.239091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f1f11-0c7f-4bcd-ac5c-1d6a8017ade4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:37.239354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f1f11-0c7f-4bcd-ac5c-1d6a8017ade4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:37.249176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f1f11-0c7f-4bcd-ac5c-1d6a8017ade4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-719f1f11-0c7f-4bcd-ac5c-1d6a8017ade4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:19:37.253925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:37.255130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:37.256296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:37.256296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:37.256296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:37.256296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:37.256296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:19:37.256296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:19:37.256296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:19:37.256296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:19:37.256296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:19:37.256296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:19:37.256296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 16:19:37.256296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:37.256296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 521/1053] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Sep 9 16:19:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:37.256296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f1f11-0c7f-4bcd-ac5c-1d6a8017ade4 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:37.271736 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f23e18-ca12-423e-a013-a02195e5cf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:37.272347 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f23e18-ca12-423e-a013-a02195e5cf57 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:37.272619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f23e18-ca12-423e-a013-a02195e5cf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:37.272880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f23e18-ca12-423e-a013-a02195e5cf57 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:37.273504 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f23e18-ca12-423e-a013-a02195e5cf57 None admin] RBAC: Adding query filter params (name=image) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:37.273796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f23e18-ca12-423e-a013-a02195e5cf57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:37.275174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f23e18-ca12-423e-a013-a02195e5cf57 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:37.280126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 533/1054] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:37.280891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f23e18-ca12-423e-a013-a02195e5cf57 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:37.283326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:37.354271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f1f11-0c7f-4bcd-ac5c-1d6a8017ade4 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:37.377999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f23e18-ca12-423e-a013-a02195e5cf57 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:37.386663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d7b9d-d1f2-4de2-a2d4-351476cb431b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:37.387294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d7b9d-d1f2-4de2-a2d4-351476cb431b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:37.387565 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d7b9d-d1f2-4de2-a2d4-351476cb431b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:37.387811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d7b9d-d1f2-4de2-a2d4-351476cb431b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:37.398134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d7b9d-d1f2-4de2-a2d4-351476cb431b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-484d7b9d-d1f2-4de2-a2d4-351476cb431b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:19:37.403497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:37.404551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:37.405378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:37.405378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:37.405378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:37.405378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:37.405378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:19:37.405378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:19:37.405378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:19:37.405378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:19:37.405378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:19:37.405378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:19:37.405378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:19:37.405378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:37.405378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 522/1055] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:19:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:37.406120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d7b9d-d1f2-4de2-a2d4-351476cb431b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:37.419693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738599a7-f89c-49f0-b710-65466ff2c058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:37.420408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738599a7-f89c-49f0-b710-65466ff2c058 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:37.420624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738599a7-f89c-49f0-b710-65466ff2c058 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:37.420896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738599a7-f89c-49f0-b710-65466ff2c058 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:37.421607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738599a7-f89c-49f0-b710-65466ff2c058 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:37.421908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738599a7-f89c-49f0-b710-65466ff2c058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:37.423336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738599a7-f89c-49f0-b710-65466ff2c058 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:37.432679 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 534/1056] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Sep 9 16:19:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:37.433476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738599a7-f89c-49f0-b710-65466ff2c058 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:37.443406 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:37.492594 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:37.509566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d7b9d-d1f2-4de2-a2d4-351476cb431b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:37.536097 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738599a7-f89c-49f0-b710-65466ff2c058 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:37.543964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40de3a8-e37c-4ad6-86ba-a5ae8ceea81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:37.544572 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40de3a8-e37c-4ad6-86ba-a5ae8ceea81c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:37.544847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40de3a8-e37c-4ad6-86ba-a5ae8ceea81c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:37.545115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40de3a8-e37c-4ad6-86ba-a5ae8ceea81c None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:37.545742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40de3a8-e37c-4ad6-86ba-a5ae8ceea81c None admin] RBAC: Adding query filter params (type=image) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:37.546041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40de3a8-e37c-4ad6-86ba-a5ae8ceea81c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:37.547456 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40de3a8-e37c-4ad6-86ba-a5ae8ceea81c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:37.552176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 523/1057] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:37.552971 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40de3a8-e37c-4ad6-86ba-a5ae8ceea81c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:37.569749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11abf5-644e-4a8a-a1ed-1e44bb3b90bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:37.570348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11abf5-644e-4a8a-a1ed-1e44bb3b90bd None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:37.570664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11abf5-644e-4a8a-a1ed-1e44bb3b90bd None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:37.570954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11abf5-644e-4a8a-a1ed-1e44bb3b90bd None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:37.571613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11abf5-644e-4a8a-a1ed-1e44bb3b90bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:37.571914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11abf5-644e-4a8a-a1ed-1e44bb3b90bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:37.573277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11abf5-644e-4a8a-a1ed-1e44bb3b90bd None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:37.581454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 535/1058] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:19:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:37.582209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11abf5-644e-4a8a-a1ed-1e44bb3b90bd None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:37.645512 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:37.650319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40de3a8-e37c-4ad6-86ba-a5ae8ceea81c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:37.680718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11abf5-644e-4a8a-a1ed-1e44bb3b90bd None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:37.685213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa3e5f-97c3-4fda-9944-565ec36ee48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:37.685841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa3e5f-97c3-4fda-9944-565ec36ee48f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:37.686119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa3e5f-97c3-4fda-9944-565ec36ee48f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:37.686384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa3e5f-97c3-4fda-9944-565ec36ee48f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:37.698410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa3e5f-97c3-4fda-9944-565ec36ee48f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:37.699885 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa3e5f-97c3-4fda-9944-565ec36ee48f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:37.707731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-69aa3e5f-97c3-4fda-9944-565ec36ee48f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:37.713922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:37.714919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:37.715792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:37.715792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:37.715792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:37.715792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 524/1059] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:37.715792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa3e5f-97c3-4fda-9944-565ec36ee48f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:37.720615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9d585-6553-4c18-a1a0-2aea42bea285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:37.721228 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9d585-6553-4c18-a1a0-2aea42bea285 None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:37.721515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9d585-6553-4c18-a1a0-2aea42bea285 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:37.721767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9d585-6553-4c18-a1a0-2aea42bea285 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:37.722429 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9d585-6553-4c18-a1a0-2aea42bea285 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:37.723940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9d585-6553-4c18-a1a0-2aea42bea285 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:37.745044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 536/1060] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:19:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:37.746050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9d585-6553-4c18-a1a0-2aea42bea285 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:37.844317 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa3e5f-97c3-4fda-9944-565ec36ee48f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:37.867153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9d585-6553-4c18-a1a0-2aea42bea285 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:37.881135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb1d4f-09a9-45cb-99b0-8dbcc9480655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:37.881785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb1d4f-09a9-45cb-99b0-8dbcc9480655 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:37.882080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb1d4f-09a9-45cb-99b0-8dbcc9480655 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:37.882329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb1d4f-09a9-45cb-99b0-8dbcc9480655 None admin] PATH_INFO: `/v3/users/d1e9019cf071449889d410e223ad6a3e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:37.896946 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb1d4f-09a9-45cb-99b0-8dbcc9480655 None admin] RBAC: Authorizing `identity:get_user(user_id=d1e9019cf071449889d410e223ad6a3e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:37.898319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb1d4f-09a9-45cb-99b0-8dbcc9480655 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:37.900086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 525/1061] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:19:37 2022] GET /identity/v3/users/d1e9019cf071449889d410e223ad6a3e => generated 285 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:37.906089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb1d4f-09a9-45cb-99b0-8dbcc9480655 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:37.912515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8541a8e-d46c-4215-971f-cff417ad52be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:37.913155 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8541a8e-d46c-4215-971f-cff417ad52be None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:37.913426 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8541a8e-d46c-4215-971f-cff417ad52be None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:37.913712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8541a8e-d46c-4215-971f-cff417ad52be None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:37.914364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8541a8e-d46c-4215-971f-cff417ad52be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:37.914698 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8541a8e-d46c-4215-971f-cff417ad52be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:37.916072 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8541a8e-d46c-4215-971f-cff417ad52be None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:37.917611 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:37.930807 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 537/1062] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:37.936313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8541a8e-d46c-4215-971f-cff417ad52be None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:37.978425 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:38.025503 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb1d4f-09a9-45cb-99b0-8dbcc9480655 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:38.041497 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8541a8e-d46c-4215-971f-cff417ad52be None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:38.063258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc24dd9-6b2e-43e0-8c07-a03617311752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:38.063878 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc24dd9-6b2e-43e0-8c07-a03617311752 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:38.064150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc24dd9-6b2e-43e0-8c07-a03617311752 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:38.064426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc24dd9-6b2e-43e0-8c07-a03617311752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:38.079381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724f9a0-e729-44ef-a07f-ae1b6cde1613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:38.080121 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724f9a0-e729-44ef-a07f-ae1b6cde1613 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:38.080463 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724f9a0-e729-44ef-a07f-ae1b6cde1613 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:38.080800 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724f9a0-e729-44ef-a07f-ae1b6cde1613 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:38.088903 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724f9a0-e729-44ef-a07f-ae1b6cde1613 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:38.090361 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724f9a0-e729-44ef-a07f-ae1b6cde1613 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-c724f9a0-e729-44ef-a07f-ae1b6cde1613 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:19:38.103407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:38.104418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:38.106392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:38.106392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:38.106392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:19:38.106392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:19:38.106392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:19:38.106392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:19:38.106392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:19:38.106392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:19:38.106392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:19:38.106392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:38.106392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 538/1063] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Sep 9 16:19:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:38.110233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724f9a0-e729-44ef-a07f-ae1b6cde1613 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:38.142221 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc24dd9-6b2e-43e0-8c07-a03617311752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:38.144198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc24dd9-6b2e-43e0-8c07-a03617311752 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:38.153381 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:38.211005 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:38.215340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724f9a0-e729-44ef-a07f-ae1b6cde1613 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:38.251136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177e3a4-3886-440c-81bd-2e6ea293eb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:38.251754 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e3a4-3886-440c-81bd-2e6ea293eb6d None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:38.252039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e3a4-3886-440c-81bd-2e6ea293eb6d None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:38.252394 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e3a4-3886-440c-81bd-2e6ea293eb6d None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:38.253034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177e3a4-3886-440c-81bd-2e6ea293eb6d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:38.253337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177e3a4-3886-440c-81bd-2e6ea293eb6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:38.254828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177e3a4-3886-440c-81bd-2e6ea293eb6d None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:38.262925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 526/1064] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:19:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 16:19:38.268392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc24dd9-6b2e-43e0-8c07-a03617311752 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:38.274547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 539/1065] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Sep 9 16:19:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:38.278202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177e3a4-3886-440c-81bd-2e6ea293eb6d None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:38.374145 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc24dd9-6b2e-43e0-8c07-a03617311752 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:38.378829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177e3a4-3886-440c-81bd-2e6ea293eb6d None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:38.383088 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:38.414585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b058f-3d53-4ce1-870b-3f5a68dc6fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:38.415142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b058f-3d53-4ce1-870b-3f5a68dc6fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:38.415419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b058f-3d53-4ce1-870b-3f5a68dc6fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:38.415687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b058f-3d53-4ce1-870b-3f5a68dc6fd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:38.426073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b058f-3d53-4ce1-870b-3f5a68dc6fd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:38.427614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b058f-3d53-4ce1-870b-3f5a68dc6fd4 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:38.429707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf33bf1-7f6d-4644-9f45-840cba390e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:38.430427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf33bf1-7f6d-4644-9f45-840cba390e26 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:38.430755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf33bf1-7f6d-4644-9f45-840cba390e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:38.431026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf33bf1-7f6d-4644-9f45-840cba390e26 None admin] PATH_INFO: `/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:38.432183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf33bf1-7f6d-4644-9f45-840cba390e26 None admin] RBAC: Authorizing `identity:get_project(project_id=ed8529c2b5544d13942a02fd1ad8f5dd)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:38.433598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf33bf1-7f6d-4644-9f45-840cba390e26 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:38.435410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 527/1066] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:19:38 2022] GET /identity/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:38.441772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf33bf1-7f6d-4644-9f45-840cba390e26 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-f73b058f-3d53-4ce1-870b-3f5a68dc6fd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:38.445715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:38.446681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:38.447561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:38.447561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:38.447561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:38.447561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 540/1067] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:19:38.452989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b058f-3d53-4ce1-870b-3f5a68dc6fd4 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:38.465907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:38.550960 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf33bf1-7f6d-4644-9f45-840cba390e26 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:38.560442 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b058f-3d53-4ce1-870b-3f5a68dc6fd4 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:38.586004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe36061-f486-474f-88b2-88e0e50e3290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:38.586636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe36061-f486-474f-88b2-88e0e50e3290 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:38.586912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe36061-f486-474f-88b2-88e0e50e3290 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:38.587183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe36061-f486-474f-88b2-88e0e50e3290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:38.587895 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe36061-f486-474f-88b2-88e0e50e3290 None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=ed8529c2b5544d13942a02fd1ad8f5dd, user.id=d1e9019cf071449889d410e223ad6a3e) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:38.588215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe36061-f486-474f-88b2-88e0e50e3290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:38.589617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe36061-f486-474f-88b2-88e0e50e3290 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:38.595248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11e23c-8606-454c-b84a-e2cabf127387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:38.595407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 528/1068] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:19:38 2022] GET /identity/v3/role_assignments?user.id=d1e9019cf071449889d410e223ad6a3e&scope.project.id=ed8529c2b5544d13942a02fd1ad8f5dd&role.id=c6da7ba2be504943af29338ee8855f05 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:38.596236 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11e23c-8606-454c-b84a-e2cabf127387 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:38.596688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11e23c-8606-454c-b84a-e2cabf127387 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:38.597128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11e23c-8606-454c-b84a-e2cabf127387 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:38.597949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11e23c-8606-454c-b84a-e2cabf127387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:38.598533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11e23c-8606-454c-b84a-e2cabf127387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:38.600145 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11e23c-8606-454c-b84a-e2cabf127387 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:38.618802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 541/1069] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:38.620614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe36061-f486-474f-88b2-88e0e50e3290 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:38.670253 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:38.742337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe36061-f486-474f-88b2-88e0e50e3290 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:38.787046 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f353b-9b16-4f1a-84c6-16958f674f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:38.787678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f353b-9b16-4f1a-84c6-16958f674f9d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:38.787956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f353b-9b16-4f1a-84c6-16958f674f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:38.788232 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f353b-9b16-4f1a-84c6-16958f674f9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:38.802097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f353b-9b16-4f1a-84c6-16958f674f9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:38.803653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f353b-9b16-4f1a-84c6-16958f674f9d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-2b2f353b-9b16-4f1a-84c6-16958f674f9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:19:38.817843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:38.819005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:38.819893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:19:38.819893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:19:38.819893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:19:38.819893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:19:38.819893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:19:38.819893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:19:38.819893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:38.819893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:38.819893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 529/1070] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:38.827639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11e23c-8606-454c-b84a-e2cabf127387 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:38.902015 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:38.936867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11e23c-8606-454c-b84a-e2cabf127387 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:38.972762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22c95e-f316-4d79-8c78-16d49e40e968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:38.973747 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22c95e-f316-4d79-8c78-16d49e40e968 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:38.974219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22c95e-f316-4d79-8c78-16d49e40e968 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:38.974746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22c95e-f316-4d79-8c78-16d49e40e968 None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:38.975643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22c95e-f316-4d79-8c78-16d49e40e968 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:38.976138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22c95e-f316-4d79-8c78-16d49e40e968 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:38.977737 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22c95e-f316-4d79-8c78-16d49e40e968 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:38.994765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 542/1071] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:19:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:38.996046 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f353b-9b16-4f1a-84c6-16958f674f9d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:39.105220 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:39.109357 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f353b-9b16-4f1a-84c6-16958f674f9d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:39.145449 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb947d4-e710-454f-96ed-1d84d9535f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:39.146120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb947d4-e710-454f-96ed-1d84d9535f92 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:39.146407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb947d4-e710-454f-96ed-1d84d9535f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:39.146705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb947d4-e710-454f-96ed-1d84d9535f92 None admin] PATH_INFO: `/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/152a41ad598b4b01b1881d928c53c848/roles/4d4fab0f77f847fbb500c58e80767d9e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:39.148890 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb947d4-e710-454f-96ed-1d84d9535f92 None admin] RBAC: Authorizing `identity:create_grant(project_id=253ff30a56444df28d44c4b7eaadb276, user_id=152a41ad598b4b01b1881d928c53c848, role_id=4d4fab0f77f847fbb500c58e80767d9e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:39.150354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb947d4-e710-454f-96ed-1d84d9535f92 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:39.169495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 530/1072] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:19:38 2022] PUT /identity/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/152a41ad598b4b01b1881d928c53c848/roles/4d4fab0f77f847fbb500c58e80767d9e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:19:39.900931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c358566-da51-42d0-aefc-19e0fa324800 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:39.901511 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c358566-da51-42d0-aefc-19e0fa324800 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:39.901511 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c358566-da51-42d0-aefc-19e0fa324800 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:39.903001 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 543/1073] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:39.908356 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4e675-30cf-4221-a0ec-c0e8876d1f9e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:39.908774 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4e675-30cf-4221-a0ec-c0e8876d1f9e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:39.908873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4e675-30cf-4221-a0ec-c0e8876d1f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:39.927646 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7bd4e675-30cf-4221-a0ec-c0e8876d1f9e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:39.928842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7bd4e675-30cf-4221-a0ec-c0e8876d1f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:39.997827 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 531/1074] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:40.259479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c358566-da51-42d0-aefc-19e0fa324800 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:40.368435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c358566-da51-42d0-aefc-19e0fa324800 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:40.394952 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:40.404107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df9e61-e34d-4de8-94f0-b3a35d9f623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:40.404726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df9e61-e34d-4de8-94f0-b3a35d9f623a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:40.405001 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df9e61-e34d-4de8-94f0-b3a35d9f623a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:40.405268 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df9e61-e34d-4de8-94f0-b3a35d9f623a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:40.471606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df9e61-e34d-4de8-94f0-b3a35d9f623a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:40.473050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df9e61-e34d-4de8-94f0-b3a35d9f623a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:40.569366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 544/1075] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:19:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:19:40.575313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4e675-30cf-4221-a0ec-c0e8876d1f9e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:40.667623 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:40.680983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4e675-30cf-4221-a0ec-c0e8876d1f9e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:40.698156 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227f261-8e2c-4dc1-ac3f-cbc2add36506 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:40.698466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227f261-8e2c-4dc1-ac3f-cbc2add36506 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:40.698745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227f261-8e2c-4dc1-ac3f-cbc2add36506 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:40.700282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 545/1076] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:19:40.704891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df81ea-ff84-4f22-b612-32e490e73908 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:40.705174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df81ea-ff84-4f22-b612-32e490e73908 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:40.705412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df81ea-ff84-4f22-b612-32e490e73908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:40.717149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751702dc-adc2-4d28-82bf-97fc057afafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:40.718423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751702dc-adc2-4d28-82bf-97fc057afafe None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:40.718729 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751702dc-adc2-4d28-82bf-97fc057afafe None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:40.718999 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751702dc-adc2-4d28-82bf-97fc057afafe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:40.723926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-69df81ea-ff84-4f22-b612-32e490e73908 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:40.725167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-69df81ea-ff84-4f22-b612-32e490e73908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:40.729675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751702dc-adc2-4d28-82bf-97fc057afafe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:40.731155 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751702dc-adc2-4d28-82bf-97fc057afafe None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-751702dc-adc2-4d28-82bf-97fc057afafe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:40.745586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:40.746669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:40.747557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:40.747557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:40.747557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:40.747557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 532/1077] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:40.752195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:40.753086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751702dc-adc2-4d28-82bf-97fc057afafe None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:40.798731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 546/1078] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:19:40.858559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751702dc-adc2-4d28-82bf-97fc057afafe None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:40.896173 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0890869-5b38-493d-acba-eec0fb6fa6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:40.896800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0890869-5b38-493d-acba-eec0fb6fa6cc None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:40.897078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0890869-5b38-493d-acba-eec0fb6fa6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:40.897325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0890869-5b38-493d-acba-eec0fb6fa6cc None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:40.897985 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0890869-5b38-493d-acba-eec0fb6fa6cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:40.898330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0890869-5b38-493d-acba-eec0fb6fa6cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:40.899738 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0890869-5b38-493d-acba-eec0fb6fa6cc None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:40.910605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:40.918600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 533/1079] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:41.075134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df81ea-ff84-4f22-b612-32e490e73908 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:41.177582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df81ea-ff84-4f22-b612-32e490e73908 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:41.214749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26746f49-382b-4ed4-abe4-30c943dc974d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:41.215424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26746f49-382b-4ed4-abe4-30c943dc974d None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:41.215691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26746f49-382b-4ed4-abe4-30c943dc974d None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:41.215957 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26746f49-382b-4ed4-abe4-30c943dc974d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:41.230150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26746f49-382b-4ed4-abe4-30c943dc974d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-26746f49-382b-4ed4-abe4-30c943dc974d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:19:41.235143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:41.237196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:41.238028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:41.238028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:41.238028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:41.238028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:41.238028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:19:41.238028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:19:41.238028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:19:41.238028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:19:41.238028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:19:41.238028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:19:41.238028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:19:41.238028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:41.238028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 547/1080] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:19:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:41.244653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0890869-5b38-493d-acba-eec0fb6fa6cc None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:41.344093 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0890869-5b38-493d-acba-eec0fb6fa6cc None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:41.347831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0a0c7-2d93-4db5-94a7-075d1e2d542b None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:41.348105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0a0c7-2d93-4db5-94a7-075d1e2d542b None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:41.348406 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0a0c7-2d93-4db5-94a7-075d1e2d542b None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:41.350349 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 548/1081] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:19:41.355273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae45ae-6817-46d4-9e3b-07d52a17cebd None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:41.355556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae45ae-6817-46d4-9e3b-07d52a17cebd None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:41.355825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae45ae-6817-46d4-9e3b-07d52a17cebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:41.373564 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f5ae45ae-6817-46d4-9e3b-07d52a17cebd None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:41.374828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f5ae45ae-6817-46d4-9e3b-07d52a17cebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:41.380745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8df63-cff7-4937-9122-11feecf707e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:41.381344 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8df63-cff7-4937-9122-11feecf707e3 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:41.381620 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8df63-cff7-4937-9122-11feecf707e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:41.381876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8df63-cff7-4937-9122-11feecf707e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:41.382530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8df63-cff7-4937-9122-11feecf707e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:41.382844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8df63-cff7-4937-9122-11feecf707e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:41.384187 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8df63-cff7-4937-9122-11feecf707e3 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:41.388128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:41.392714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 534/1082] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:19:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:41.398656 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8df63-cff7-4937-9122-11feecf707e3 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:41.445670 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 549/1083] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:19:41.457885 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:41.495453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8df63-cff7-4937-9122-11feecf707e3 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:41.529421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55ca54-2219-4f0a-8736-e5650726bba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:41.530061 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55ca54-2219-4f0a-8736-e5650726bba5 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:41.530335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55ca54-2219-4f0a-8736-e5650726bba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:41.530635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55ca54-2219-4f0a-8736-e5650726bba5 None admin] PATH_INFO: `/v3/users/d1e9019cf071449889d410e223ad6a3e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:41.531899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55ca54-2219-4f0a-8736-e5650726bba5 None admin] RBAC: Authorizing `identity:get_user(user_id=d1e9019cf071449889d410e223ad6a3e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:41.533448 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55ca54-2219-4f0a-8736-e5650726bba5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:41.535074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 535/1084] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:19:41 2022] GET /identity/v3/users/d1e9019cf071449889d410e223ad6a3e => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:41.541538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae45ae-6817-46d4-9e3b-07d52a17cebd None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:41.644045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae45ae-6817-46d4-9e3b-07d52a17cebd None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:41.681547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e596d49-2f2f-4222-96bf-e182197f7d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:41.682182 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596d49-2f2f-4222-96bf-e182197f7d1f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:41.682458 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596d49-2f2f-4222-96bf-e182197f7d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:41.682757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596d49-2f2f-4222-96bf-e182197f7d1f None admin] PATH_INFO: `/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:41.683870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e596d49-2f2f-4222-96bf-e182197f7d1f None admin] RBAC: Authorizing `identity:get_project(project_id=ed8529c2b5544d13942a02fd1ad8f5dd)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:41.685310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e596d49-2f2f-4222-96bf-e182197f7d1f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:41.686979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 550/1085] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:19:41 2022] GET /identity/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:41.693558 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55ca54-2219-4f0a-8736-e5650726bba5 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:41.706083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e596d49-2f2f-4222-96bf-e182197f7d1f None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:41.724507 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:41.793994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55ca54-2219-4f0a-8736-e5650726bba5 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:41.804625 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e596d49-2f2f-4222-96bf-e182197f7d1f None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:41.822768 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:41.829067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f23cb7-6ec6-4ed8-990a-60ea621ae8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:41.829733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f23cb7-6ec6-4ed8-990a-60ea621ae8d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:41.830010 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f23cb7-6ec6-4ed8-990a-60ea621ae8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:41.830279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f23cb7-6ec6-4ed8-990a-60ea621ae8d0 None admin] PATH_INFO: `/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd/users/d1e9019cf071449889d410e223ad6a3e/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:41.832214 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f23cb7-6ec6-4ed8-990a-60ea621ae8d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed8529c2b5544d13942a02fd1ad8f5dd, user_id=d1e9019cf071449889d410e223ad6a3e, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:41.833681 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f23cb7-6ec6-4ed8-990a-60ea621ae8d0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:41.839371 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b521e-264d-4aa0-9432-c5b71041f76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:41.840142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b521e-264d-4aa0-9432-c5b71041f76e None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:41.840433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b521e-264d-4aa0-9432-c5b71041f76e None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:41.840744 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b521e-264d-4aa0-9432-c5b71041f76e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:41.847161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 536/1086] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:19:41 2022] PUT /identity/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd/users/d1e9019cf071449889d410e223ad6a3e/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:19:41.856790 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b521e-264d-4aa0-9432-c5b71041f76e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-a31b521e-264d-4aa0-9432-c5b71041f76e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:19:41.862290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:41.863360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:41.864360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:41.864360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:41.864360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:41.864360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:41.864360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:19:41.864360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:19:41.864360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:19:41.864360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:19:41.864360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:19:41.864360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:19:41.864360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 16:19:41.864360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:41.864360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 551/1087] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Sep 9 16:19:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:41.870029 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f23cb7-6ec6-4ed8-990a-60ea621ae8d0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:41.980835 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f23cb7-6ec6-4ed8-990a-60ea621ae8d0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:42.020343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ab2f3-10e2-44ef-a247-cfb428c511b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:42.020974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ab2f3-10e2-44ef-a247-cfb428c511b7 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:42.021249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ab2f3-10e2-44ef-a247-cfb428c511b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:42.021479 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ab2f3-10e2-44ef-a247-cfb428c511b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:42.022174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ab2f3-10e2-44ef-a247-cfb428c511b7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:42.022475 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ab2f3-10e2-44ef-a247-cfb428c511b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:42.023885 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ab2f3-10e2-44ef-a247-cfb428c511b7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:42.033336 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 537/1088] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Sep 9 16:19:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:42.039480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b521e-264d-4aa0-9432-c5b71041f76e None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:42.119330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:42.142355 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b521e-264d-4aa0-9432-c5b71041f76e None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:42.177146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998d3b8-e359-4cbe-9767-ce689507715b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:42.177780 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998d3b8-e359-4cbe-9767-ce689507715b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:42.178020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998d3b8-e359-4cbe-9767-ce689507715b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:42.178314 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998d3b8-e359-4cbe-9767-ce689507715b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:42.188863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998d3b8-e359-4cbe-9767-ce689507715b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:42.190692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998d3b8-e359-4cbe-9767-ce689507715b None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-b998d3b8-e359-4cbe-9767-ce689507715b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:42.205388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:42.206627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:42.207526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:42.207526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:42.207526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:42.207526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 552/1089] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:42.212566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ab2f3-10e2-44ef-a247-cfb428c511b7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:42.266040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:42.315268 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ab2f3-10e2-44ef-a247-cfb428c511b7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:42.350605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d7442-0232-4664-985b-8d0a5bcc1745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:42.351207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d7442-0232-4664-985b-8d0a5bcc1745 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:42.351480 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d7442-0232-4664-985b-8d0a5bcc1745 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:42.351743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d7442-0232-4664-985b-8d0a5bcc1745 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:42.352359 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d7442-0232-4664-985b-8d0a5bcc1745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:42.352662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d7442-0232-4664-985b-8d0a5bcc1745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:42.354014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d7442-0232-4664-985b-8d0a5bcc1745 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:42.365251 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 538/1090] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:42.371330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998d3b8-e359-4cbe-9767-ce689507715b None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:42.477050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998d3b8-e359-4cbe-9767-ce689507715b None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:42.506244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:42.514753 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fabd2-c90c-4d3f-8649-fec0cfb7daa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:42.515389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fabd2-c90c-4d3f-8649-fec0cfb7daa8 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:42.515661 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fabd2-c90c-4d3f-8649-fec0cfb7daa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:42.515912 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fabd2-c90c-4d3f-8649-fec0cfb7daa8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:42.523356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fabd2-c90c-4d3f-8649-fec0cfb7daa8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:42.524727 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fabd2-c90c-4d3f-8649-fec0cfb7daa8 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-907fabd2-c90c-4d3f-8649-fec0cfb7daa8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:19:42.535914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:42.536961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:42.537970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:42.537970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:42.537970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:19:42.537970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:19:42.537970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:19:42.537970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:19:42.537970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:19:42.537970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:19:42.537970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 16:19:42.537970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:42.537970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 553/1091] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Sep 9 16:19:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:42.543217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d7442-0232-4664-985b-8d0a5bcc1745 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:42.656910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d7442-0232-4664-985b-8d0a5bcc1745 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:42.669594 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:42.692701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c92fd-922c-467b-a1ed-350e7940e1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:42.693308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c92fd-922c-467b-a1ed-350e7940e1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:42.693584 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c92fd-922c-467b-a1ed-350e7940e1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:42.693824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c92fd-922c-467b-a1ed-350e7940e1b8 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:42.694551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c92fd-922c-467b-a1ed-350e7940e1b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:42.694864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c92fd-922c-467b-a1ed-350e7940e1b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:42.696261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c92fd-922c-467b-a1ed-350e7940e1b8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:42.715651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 539/1092] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Sep 9 16:19:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:42.721552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fabd2-c90c-4d3f-8649-fec0cfb7daa8 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:42.819746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fabd2-c90c-4d3f-8649-fec0cfb7daa8 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:42.852940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962941f8-f256-4419-821d-942204057ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:42.853536 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962941f8-f256-4419-821d-942204057ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:42.853811 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962941f8-f256-4419-821d-942204057ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:42.854082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962941f8-f256-4419-821d-942204057ed9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:42.909749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:42.923516 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962941f8-f256-4419-821d-942204057ed9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:42.924977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962941f8-f256-4419-821d-942204057ed9 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:43.019638 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 554/1093] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:19:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:19:43.025635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c92fd-922c-467b-a1ed-350e7940e1b8 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:43.130625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c92fd-922c-467b-a1ed-350e7940e1b8 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:43.164171 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34e6d4-ccf2-4e75-bd75-03ac59b3734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:43.164793 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34e6d4-ccf2-4e75-bd75-03ac59b3734c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:43.165064 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34e6d4-ccf2-4e75-bd75-03ac59b3734c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:43.165327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34e6d4-ccf2-4e75-bd75-03ac59b3734c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:43.175448 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34e6d4-ccf2-4e75-bd75-03ac59b3734c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:43.176912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34e6d4-ccf2-4e75-bd75-03ac59b3734c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-ef34e6d4-ccf2-4e75-bd75-03ac59b3734c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:19:43.191661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:43.192781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:19:43.194067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:19:43.194067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:19:43.194067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:43.194067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 540/1094] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:43.198728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962941f8-f256-4419-821d-942204057ed9 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:43.280471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:43.293684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962941f8-f256-4419-821d-942204057ed9 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:43.326811 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2c599-b53b-4eeb-8dd8-9d120b48e5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:43.327396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2c599-b53b-4eeb-8dd8-9d120b48e5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:43.327672 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2c599-b53b-4eeb-8dd8-9d120b48e5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:43.327943 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2c599-b53b-4eeb-8dd8-9d120b48e5b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:43.328656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2c599-b53b-4eeb-8dd8-9d120b48e5b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:43.328961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2c599-b53b-4eeb-8dd8-9d120b48e5b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:43.330336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2c599-b53b-4eeb-8dd8-9d120b48e5b0 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:43.341541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 555/1095] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:43.347367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34e6d4-ccf2-4e75-bd75-03ac59b3734c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:43.374100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:43.450621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34e6d4-ccf2-4e75-bd75-03ac59b3734c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:43.484635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea23b1e-28ab-445b-b381-b25e24c88603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:43.485259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea23b1e-28ab-445b-b381-b25e24c88603 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:43.485529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea23b1e-28ab-445b-b381-b25e24c88603 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:43.485790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea23b1e-28ab-445b-b381-b25e24c88603 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:43.492630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea23b1e-28ab-445b-b381-b25e24c88603 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:43.494065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea23b1e-28ab-445b-b381-b25e24c88603 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-eea23b1e-28ab-445b-b381-b25e24c88603 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:19:43.505228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:43.506135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:43.507435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:19:43.507435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:19:43.507435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:19:43.507435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:19:43.507435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:19:43.507435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:19:43.507435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:19:43.507435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:19:43.507435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 541/1096] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:19:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:43.512551 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2c599-b53b-4eeb-8dd8-9d120b48e5b0 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:43.612372 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2c599-b53b-4eeb-8dd8-9d120b48e5b0 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:43.646018 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f9578-8817-4e6f-aaa5-efc05fe677ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:43.646650 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f9578-8817-4e6f-aaa5-efc05fe677ad None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:43.646910 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f9578-8817-4e6f-aaa5-efc05fe677ad None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:43.647177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f9578-8817-4e6f-aaa5-efc05fe677ad None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:43.647857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f9578-8817-4e6f-aaa5-efc05fe677ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:43.648154 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f9578-8817-4e6f-aaa5-efc05fe677ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:43.649568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f9578-8817-4e6f-aaa5-efc05fe677ad None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:43.661472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 556/1097] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:19:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:43.667663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea23b1e-28ab-445b-b381-b25e24c88603 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:43.749769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:43.768084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea23b1e-28ab-445b-b381-b25e24c88603 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:43.802094 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93ce68-af6a-46aa-baf8-368cb80d29e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:43.802724 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93ce68-af6a-46aa-baf8-368cb80d29e4 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:43.802972 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93ce68-af6a-46aa-baf8-368cb80d29e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:43.803261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93ce68-af6a-46aa-baf8-368cb80d29e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:43.803969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93ce68-af6a-46aa-baf8-368cb80d29e4 None admin] RBAC: Adding query filter params (role.id=4d4fab0f77f847fbb500c58e80767d9e, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=152a41ad598b4b01b1881d928c53c848) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:43.804246 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93ce68-af6a-46aa-baf8-368cb80d29e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:43.805712 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93ce68-af6a-46aa-baf8-368cb80d29e4 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:43.811697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 542/1098] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:19:43 2022] GET /identity/v3/role_assignments?user.id=152a41ad598b4b01b1881d928c53c848&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=4d4fab0f77f847fbb500c58e80767d9e => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:43.977227 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbebda7-8a78-491e-a8a6-60ba35eeba72 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:43.977444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbebda7-8a78-491e-a8a6-60ba35eeba72 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:43.977647 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbebda7-8a78-491e-a8a6-60ba35eeba72 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:43.979032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 557/1099] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:43.985701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7343a3-96ba-48e2-9359-71c64fa24c11 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:43.985984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7343a3-96ba-48e2-9359-71c64fa24c11 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:43.986255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7343a3-96ba-48e2-9359-71c64fa24c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:44.006917 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-6a7343a3-96ba-48e2-9359-71c64fa24c11 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:44.008114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6a7343a3-96ba-48e2-9359-71c64fa24c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:44.086827 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 543/1100] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:44.342031 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbebda7-8a78-491e-a8a6-60ba35eeba72 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:44.353634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:44.469429 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbebda7-8a78-491e-a8a6-60ba35eeba72 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:44.508900 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e079d-af99-4e28-b92a-017acff9fcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:44.509544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e079d-af99-4e28-b92a-017acff9fcbe None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:44.509831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e079d-af99-4e28-b92a-017acff9fcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:44.510098 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e079d-af99-4e28-b92a-017acff9fcbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:44.521771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e079d-af99-4e28-b92a-017acff9fcbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-b72e079d-af99-4e28-b92a-017acff9fcbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:19:44.526817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:19:44.527774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:19:44.528606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:19:44.528606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:19:44.528606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:19:44.528606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:19:44.528606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:19:44.528606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:19:44.528606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:19:44.528606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:19:44.528606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:19:44.528606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:19:44.528606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:19:44.528606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:19:44.528606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 558/1101] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:19:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:19:44.534491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7343a3-96ba-48e2-9359-71c64fa24c11 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:44.626586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:44.654704 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7343a3-96ba-48e2-9359-71c64fa24c11 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:44.688090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab01906-33d9-48f0-bc91-9e037f1fb84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:44.688704 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab01906-33d9-48f0-bc91-9e037f1fb84b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:44.688976 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab01906-33d9-48f0-bc91-9e037f1fb84b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:44.689241 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab01906-33d9-48f0-bc91-9e037f1fb84b None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:44.690259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab01906-33d9-48f0-bc91-9e037f1fb84b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:44.690583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab01906-33d9-48f0-bc91-9e037f1fb84b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:44.692086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab01906-33d9-48f0-bc91-9e037f1fb84b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:44.700388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 544/1102] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:19:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:44.706871 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e079d-af99-4e28-b92a-017acff9fcbe None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:44.770046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:44.809275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e079d-af99-4e28-b92a-017acff9fcbe None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:44.844913 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6920c2f-828b-45ef-92f5-0423100536b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:44.845533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6920c2f-828b-45ef-92f5-0423100536b1 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:44.845814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6920c2f-828b-45ef-92f5-0423100536b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:44.846086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6920c2f-828b-45ef-92f5-0423100536b1 None admin] PATH_INFO: `/v3/users/d1e9019cf071449889d410e223ad6a3e` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:44.847283 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6920c2f-828b-45ef-92f5-0423100536b1 None admin] RBAC: Authorizing `identity:get_user(user_id=d1e9019cf071449889d410e223ad6a3e)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:44.848768 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6920c2f-828b-45ef-92f5-0423100536b1 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:44.850303 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 559/1103] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:19:44 2022] GET /identity/v3/users/d1e9019cf071449889d410e223ad6a3e => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:44.856940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab01906-33d9-48f0-bc91-9e037f1fb84b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:44.959316 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab01906-33d9-48f0-bc91-9e037f1fb84b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:44.993458 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e0f1f-929d-4ab0-9f8b-fbecab98d1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:44.994051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e0f1f-929d-4ab0-9f8b-fbecab98d1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:44.994327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e0f1f-929d-4ab0-9f8b-fbecab98d1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:44.994616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e0f1f-929d-4ab0-9f8b-fbecab98d1c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:45.003261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:45.063263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e0f1f-929d-4ab0-9f8b-fbecab98d1c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:45.065057 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e0f1f-929d-4ab0-9f8b-fbecab98d1c9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:45.164560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 545/1104] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:19:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:19:45.171131 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6920c2f-828b-45ef-92f5-0423100536b1 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:45.279852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6920c2f-828b-45ef-92f5-0423100536b1 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:45.314200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45284ea7-0f37-445b-865d-5a390fc709a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:45.315213 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45284ea7-0f37-445b-865d-5a390fc709a6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:45.315488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45284ea7-0f37-445b-865d-5a390fc709a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:45.315759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45284ea7-0f37-445b-865d-5a390fc709a6 None admin] PATH_INFO: `/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:45.316997 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45284ea7-0f37-445b-865d-5a390fc709a6 None admin] RBAC: Authorizing `identity:get_project(project_id=ed8529c2b5544d13942a02fd1ad8f5dd)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:45.318475 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45284ea7-0f37-445b-865d-5a390fc709a6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:45.320114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 560/1105] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:19:45 2022] GET /identity/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:45.327070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e0f1f-929d-4ab0-9f8b-fbecab98d1c9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:45.375668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:45.435839 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e0f1f-929d-4ab0-9f8b-fbecab98d1c9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:45.470877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bf00b-057b-4a05-90e2-669ba89b4b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:45.471459 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bf00b-057b-4a05-90e2-669ba89b4b9c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:45.471730 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bf00b-057b-4a05-90e2-669ba89b4b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:45.471994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bf00b-057b-4a05-90e2-669ba89b4b9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:45.472698 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bf00b-057b-4a05-90e2-669ba89b4b9c None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=ed8529c2b5544d13942a02fd1ad8f5dd, user.id=d1e9019cf071449889d410e223ad6a3e) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:45.473003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bf00b-057b-4a05-90e2-669ba89b4b9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:45.474484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bf00b-057b-4a05-90e2-669ba89b4b9c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:45.480248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 546/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:19:45 2022] GET /identity/v3/role_assignments?user.id=d1e9019cf071449889d410e223ad6a3e&scope.project.id=ed8529c2b5544d13942a02fd1ad8f5dd&role.id=c6da7ba2be504943af29338ee8855f05 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:47.657177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed89e0-f6e8-490c-ac25-f8707ae3ca5b None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:47.657771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed89e0-f6e8-490c-ac25-f8707ae3ca5b None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:47.657771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed89e0-f6e8-490c-ac25-f8707ae3ca5b None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:47.659110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 561/1107] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:47.665802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a8b37-c2ed-4f6e-ab9d-92e539b77690 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:47.666564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a8b37-c2ed-4f6e-ab9d-92e539b77690 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:47.666729 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a8b37-c2ed-4f6e-ab9d-92e539b77690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:47.695762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b71a8b37-c2ed-4f6e-ab9d-92e539b77690 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:47.697319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b71a8b37-c2ed-4f6e-ab9d-92e539b77690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:47.765655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 547/1108] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:48.040194 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed89e0-f6e8-490c-ac25-f8707ae3ca5b None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:48.044642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:48.151008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed89e0-f6e8-490c-ac25-f8707ae3ca5b None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:48.186128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2778d-23d2-4592-9c1a-8e35d1e8f09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:48.186773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2778d-23d2-4592-9c1a-8e35d1e8f09d None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:48.187044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2778d-23d2-4592-9c1a-8e35d1e8f09d None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:48.187311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2778d-23d2-4592-9c1a-8e35d1e8f09d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:48.188519 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2778d-23d2-4592-9c1a-8e35d1e8f09d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:48.190306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2778d-23d2-4592-9c1a-8e35d1e8f09d None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:48.191916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 562/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:19:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:48.198795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a8b37-c2ed-4f6e-ab9d-92e539b77690 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:48.237047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:48.307742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a8b37-c2ed-4f6e-ab9d-92e539b77690 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:48.347373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750159e4-88ad-49fe-aede-2101fafdce85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:48.347969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750159e4-88ad-49fe-aede-2101fafdce85 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:48.348298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750159e4-88ad-49fe-aede-2101fafdce85 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:48.348563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750159e4-88ad-49fe-aede-2101fafdce85 None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:48.349202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750159e4-88ad-49fe-aede-2101fafdce85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:48.350658 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750159e4-88ad-49fe-aede-2101fafdce85 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:48.379413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 548/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:19:48 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:19:50.515524 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298012d-3eed-41fa-9bd0-47a3a784fbe6 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:50.516091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298012d-3eed-41fa-9bd0-47a3a784fbe6 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:50.516091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298012d-3eed-41fa-9bd0-47a3a784fbe6 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:50.517388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 563/1111] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:50.523893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a97e5-d723-469c-b8e9-ba484f4459fd None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:50.524582 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a97e5-d723-469c-b8e9-ba484f4459fd None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:50.524725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a97e5-d723-469c-b8e9-ba484f4459fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:50.547734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-461a97e5-d723-469c-b8e9-ba484f4459fd None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:50.548944 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-461a97e5-d723-469c-b8e9-ba484f4459fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:50.618679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 549/1112] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:50.877423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298012d-3eed-41fa-9bd0-47a3a784fbe6 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:50.960930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:50.992194 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298012d-3eed-41fa-9bd0-47a3a784fbe6 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:51.026794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f1165-788f-4179-9a18-d80ac6470ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:51.027412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f1165-788f-4179-9a18-d80ac6470ddb None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:51.027713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f1165-788f-4179-9a18-d80ac6470ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:51.027981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f1165-788f-4179-9a18-d80ac6470ddb None admin] PATH_INFO: `/v3/roles/0d40b3176f78460f9c4a29c1e9ef843e` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:51.029414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f1165-788f-4179-9a18-d80ac6470ddb None admin] RBAC: Authorizing `identity:get_role(role_id=0d40b3176f78460f9c4a29c1e9ef843e)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:51.030825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f1165-788f-4179-9a18-d80ac6470ddb None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:51.032130 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 564/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:19:50 2022] GET /identity/v3/roles/0d40b3176f78460f9c4a29c1e9ef843e => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:51.040896 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a97e5-d723-469c-b8e9-ba484f4459fd None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:51.061151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:51.145217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a97e5-d723-469c-b8e9-ba484f4459fd None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:51.189762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c90f18-f7cd-450e-85bb-2162ed780bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:51.190393 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c90f18-f7cd-450e-85bb-2162ed780bbf None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:51.190697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c90f18-f7cd-450e-85bb-2162ed780bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:51.190964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c90f18-f7cd-450e-85bb-2162ed780bbf None admin] PATH_INFO: `/v3/users/277e1a94643a4333a090b7a59cc12f61` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:51.204265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c90f18-f7cd-450e-85bb-2162ed780bbf None admin] RBAC: Authorizing `identity:get_user(user_id=277e1a94643a4333a090b7a59cc12f61)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:51.205710 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c90f18-f7cd-450e-85bb-2162ed780bbf None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:51.207333 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 550/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:19:51 2022] GET /identity/v3/users/277e1a94643a4333a090b7a59cc12f61 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:51.215909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f1165-788f-4179-9a18-d80ac6470ddb None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:51.325873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f1165-788f-4179-9a18-d80ac6470ddb None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:51.361375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d5d77-b366-48ad-8c30-fef96ede43a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:51.361977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d5d77-b366-48ad-8c30-fef96ede43a5 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:51.362255 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d5d77-b366-48ad-8c30-fef96ede43a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:51.362543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d5d77-b366-48ad-8c30-fef96ede43a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:51.372953 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:51.431221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d5d77-b366-48ad-8c30-fef96ede43a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:51.432597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d5d77-b366-48ad-8c30-fef96ede43a5 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:51.531888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 565/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:19:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:19:51.538437 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c90f18-f7cd-450e-85bb-2162ed780bbf None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:51.624140 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:51.655476 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c90f18-f7cd-450e-85bb-2162ed780bbf None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:51.691954 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d435431-c52b-4220-8f5d-983a065e389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:51.692506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d435431-c52b-4220-8f5d-983a065e389a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:51.692820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d435431-c52b-4220-8f5d-983a065e389a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:51.693077 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d435431-c52b-4220-8f5d-983a065e389a None admin] PATH_INFO: `/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:51.694307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d435431-c52b-4220-8f5d-983a065e389a None admin] RBAC: Authorizing `identity:get_project(project_id=ed8529c2b5544d13942a02fd1ad8f5dd)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:51.695788 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d435431-c52b-4220-8f5d-983a065e389a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:51.697411 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 551/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:19:51 2022] GET /identity/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:51.704125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d5d77-b366-48ad-8c30-fef96ede43a5 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:51.748903 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:51.815772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d5d77-b366-48ad-8c30-fef96ede43a5 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:51.857073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924101a5-4b30-4759-896d-676960a7b0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:51.857946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924101a5-4b30-4759-896d-676960a7b0bb None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:51.858306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924101a5-4b30-4759-896d-676960a7b0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:51.858623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924101a5-4b30-4759-896d-676960a7b0bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:51.859425 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924101a5-4b30-4759-896d-676960a7b0bb None admin] RBAC: Adding query filter params (role.id=0d40b3176f78460f9c4a29c1e9ef843e, scope.project.id=ed8529c2b5544d13942a02fd1ad8f5dd, user.id=277e1a94643a4333a090b7a59cc12f61) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:51.859728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924101a5-4b30-4759-896d-676960a7b0bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:51.861160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924101a5-4b30-4759-896d-676960a7b0bb None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:51.866970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 566/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:19:51 2022] GET /identity/v3/role_assignments?user.id=277e1a94643a4333a090b7a59cc12f61&scope.project.id=ed8529c2b5544d13942a02fd1ad8f5dd&role.id=0d40b3176f78460f9c4a29c1e9ef843e => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:53.932726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57edfa1-8df1-46c4-85a4-d72c03957018 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:53.933641 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57edfa1-8df1-46c4-85a4-d72c03957018 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:53.933641 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57edfa1-8df1-46c4-85a4-d72c03957018 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:53.935395 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 552/1118] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:53.941285 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3d790-3388-44e8-b503-751ac1ab0000 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:53.941766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3d790-3388-44e8-b503-751ac1ab0000 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:53.941874 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3d790-3388-44e8-b503-751ac1ab0000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:53.961724 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-95e3d790-3388-44e8-b503-751ac1ab0000 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:53.962927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-95e3d790-3388-44e8-b503-751ac1ab0000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:54.032999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 567/1119] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:54.294891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57edfa1-8df1-46c4-85a4-d72c03957018 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:54.413684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57edfa1-8df1-46c4-85a4-d72c03957018 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:54.449501 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e14ab8-1616-40d0-8172-19886b4a9448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:54.450132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e14ab8-1616-40d0-8172-19886b4a9448 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:54.450409 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e14ab8-1616-40d0-8172-19886b4a9448 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:54.450710 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e14ab8-1616-40d0-8172-19886b4a9448 None admin] PATH_INFO: `/v3/roles/0d40b3176f78460f9c4a29c1e9ef843e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:54.452348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e14ab8-1616-40d0-8172-19886b4a9448 None admin] RBAC: Authorizing `identity:get_role(role_id=0d40b3176f78460f9c4a29c1e9ef843e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:54.453737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e14ab8-1616-40d0-8172-19886b4a9448 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:54.455117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 553/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:19:54 2022] GET /identity/v3/roles/0d40b3176f78460f9c4a29c1e9ef843e => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:54.461969 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3d790-3388-44e8-b503-751ac1ab0000 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:54.498956 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:54.570830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3d790-3388-44e8-b503-751ac1ab0000 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:54.608702 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7e553-e945-4ca9-bed7-13a8f6da022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:54.609601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7e553-e945-4ca9-bed7-13a8f6da022c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:54.609952 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7e553-e945-4ca9-bed7-13a8f6da022c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:54.610299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7e553-e945-4ca9-bed7-13a8f6da022c None admin] PATH_INFO: `/v3/users/277e1a94643a4333a090b7a59cc12f61` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:54.612357 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7e553-e945-4ca9-bed7-13a8f6da022c None admin] RBAC: Authorizing `identity:get_user(user_id=277e1a94643a4333a090b7a59cc12f61)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:54.614434 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7e553-e945-4ca9-bed7-13a8f6da022c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:54.616014 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 568/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:19:54 2022] GET /identity/v3/users/277e1a94643a4333a090b7a59cc12f61 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:54.624408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e14ab8-1616-40d0-8172-19886b4a9448 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:54.647353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:54.746106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e14ab8-1616-40d0-8172-19886b4a9448 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:54.788247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3939517-4b3b-4f2a-a1b4-c304e958f193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:54.788890 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3939517-4b3b-4f2a-a1b4-c304e958f193 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:54.789157 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3939517-4b3b-4f2a-a1b4-c304e958f193 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:54.789417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3939517-4b3b-4f2a-a1b4-c304e958f193 None admin] PATH_INFO: `/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:54.790692 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3939517-4b3b-4f2a-a1b4-c304e958f193 None admin] RBAC: Authorizing `identity:get_project(project_id=ed8529c2b5544d13942a02fd1ad8f5dd)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:54.792116 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3939517-4b3b-4f2a-a1b4-c304e958f193 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:54.793730 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 554/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:19:54 2022] GET /identity/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:54.802918 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7e553-e945-4ca9-bed7-13a8f6da022c None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:54.906749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7e553-e945-4ca9-bed7-13a8f6da022c None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:54.908353 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:54.943498 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867f25d-221c-4490-819f-b6ee769b4945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:54.944111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867f25d-221c-4490-819f-b6ee769b4945 None admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:54.944404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867f25d-221c-4490-819f-b6ee769b4945 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:54.944669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867f25d-221c-4490-819f-b6ee769b4945 None admin] PATH_INFO: `/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd/users/277e1a94643a4333a090b7a59cc12f61/roles/0d40b3176f78460f9c4a29c1e9ef843e` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:54.946988 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867f25d-221c-4490-819f-b6ee769b4945 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed8529c2b5544d13942a02fd1ad8f5dd, user_id=277e1a94643a4333a090b7a59cc12f61, role_id=0d40b3176f78460f9c4a29c1e9ef843e)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:54.948424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867f25d-221c-4490-819f-b6ee769b4945 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:54.963592 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 569/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:19:54 2022] PUT /identity/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd/users/277e1a94643a4333a090b7a59cc12f61/roles/0d40b3176f78460f9c4a29c1e9ef843e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:19:57.064436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccde2e-a00e-4c25-9a27-58e9d4c54810 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:57.065306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccde2e-a00e-4c25-9a27-58e9d4c54810 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:57.065306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccde2e-a00e-4c25-9a27-58e9d4c54810 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:57.067688 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 555/1124] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:19:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:19:57.075551 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073ebfc-ae9d-402b-95f2-30492240c955 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:57.075992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073ebfc-ae9d-402b-95f2-30492240c955 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:57.076116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073ebfc-ae9d-402b-95f2-30492240c955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:57.098030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-9073ebfc-ae9d-402b-95f2-30492240c955 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:19:57.099274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-9073ebfc-ae9d-402b-95f2-30492240c955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:19:57.173093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 570/1125] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:19:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:19:57.433804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbccde2e-a00e-4c25-9a27-58e9d4c54810 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:57.526554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:57.550042 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbccde2e-a00e-4c25-9a27-58e9d4c54810 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:57.585045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2145a0-e364-4dc5-8c5a-aa5251e23da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:57.585667 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2145a0-e364-4dc5-8c5a-aa5251e23da6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:57.585942 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2145a0-e364-4dc5-8c5a-aa5251e23da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:57.586211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2145a0-e364-4dc5-8c5a-aa5251e23da6 None admin] PATH_INFO: `/v3/roles/0d40b3176f78460f9c4a29c1e9ef843e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:57.587661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2145a0-e364-4dc5-8c5a-aa5251e23da6 None admin] RBAC: Authorizing `identity:get_role(role_id=0d40b3176f78460f9c4a29c1e9ef843e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:57.589039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2145a0-e364-4dc5-8c5a-aa5251e23da6 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:57.590345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 556/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:19:57 2022] GET /identity/v3/roles/0d40b3176f78460f9c4a29c1e9ef843e => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:57.596373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073ebfc-ae9d-402b-95f2-30492240c955 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:57.689948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:57.699628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073ebfc-ae9d-402b-95f2-30492240c955 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:57.733074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16ea24-573f-46bd-9c83-3e6c7d4909af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:57.733691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16ea24-573f-46bd-9c83-3e6c7d4909af None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:57.733965 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16ea24-573f-46bd-9c83-3e6c7d4909af None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:57.734235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16ea24-573f-46bd-9c83-3e6c7d4909af None admin] PATH_INFO: `/v3/users/277e1a94643a4333a090b7a59cc12f61` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:57.735425 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16ea24-573f-46bd-9c83-3e6c7d4909af None admin] RBAC: Authorizing `identity:get_user(user_id=277e1a94643a4333a090b7a59cc12f61)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:57.736855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16ea24-573f-46bd-9c83-3e6c7d4909af None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:57.738436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 571/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:19:57 2022] GET /identity/v3/users/277e1a94643a4333a090b7a59cc12f61 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:19:57.744454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2145a0-e364-4dc5-8c5a-aa5251e23da6 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:57.850884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2145a0-e364-4dc5-8c5a-aa5251e23da6 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:57.886878 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492e949-40c6-4592-9316-771d3ec01a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:57.887481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492e949-40c6-4592-9316-771d3ec01a62 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:57.887754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492e949-40c6-4592-9316-771d3ec01a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:57.888019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492e949-40c6-4592-9316-771d3ec01a62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:57.906400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:57.965194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492e949-40c6-4592-9316-771d3ec01a62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:57.966616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492e949-40c6-4592-9316-771d3ec01a62 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:58.070624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 557/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:19:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:19:58.077042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16ea24-573f-46bd-9c83-3e6c7d4909af None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:58.193127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16ea24-573f-46bd-9c83-3e6c7d4909af None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:58.230717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8951f-0be2-45a8-9bf3-d400f345f893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:58.231281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8951f-0be2-45a8-9bf3-d400f345f893 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:58.231562 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8951f-0be2-45a8-9bf3-d400f345f893 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:58.231834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8951f-0be2-45a8-9bf3-d400f345f893 None admin] PATH_INFO: `/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:58.233054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8951f-0be2-45a8-9bf3-d400f345f893 None admin] RBAC: Authorizing `identity:get_project(project_id=ed8529c2b5544d13942a02fd1ad8f5dd)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:58.234527 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8951f-0be2-45a8-9bf3-d400f345f893 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:58.236157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 572/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:19:58 2022] GET /identity/v3/projects/ed8529c2b5544d13942a02fd1ad8f5dd => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:19:58.243236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492e949-40c6-4592-9316-771d3ec01a62 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:19:58.295670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:19:58.350970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492e949-40c6-4592-9316-771d3ec01a62 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:19:58.392530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e1bb08-8d46-423a-ae21-bb1e841c66d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:19:58.393422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1bb08-8d46-423a-ae21-bb1e841c66d1 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:19:58.393824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1bb08-8d46-423a-ae21-bb1e841c66d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:19:58.394241 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1bb08-8d46-423a-ae21-bb1e841c66d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:19:58.395358 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e1bb08-8d46-423a-ae21-bb1e841c66d1 None admin] RBAC: Adding query filter params (role.id=0d40b3176f78460f9c4a29c1e9ef843e, scope.project.id=ed8529c2b5544d13942a02fd1ad8f5dd, user.id=277e1a94643a4333a090b7a59cc12f61) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:19:58.395674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e1bb08-8d46-423a-ae21-bb1e841c66d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:19:58.397208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e1bb08-8d46-423a-ae21-bb1e841c66d1 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:19:58.403279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 558/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:19:58 2022] GET /identity/v3/role_assignments?user.id=277e1a94643a4333a090b7a59cc12f61&scope.project.id=ed8529c2b5544d13942a02fd1ad8f5dd&role.id=0d40b3176f78460f9c4a29c1e9ef843e => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:20:00.557322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90c054-2835-43e9-bf43-878c4ffed54c None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:00.557890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90c054-2835-43e9-bf43-878c4ffed54c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:00.557890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90c054-2835-43e9-bf43-878c4ffed54c None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:00.559248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 573/1131] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:20:00.566138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99290c18-cb62-45e0-9f10-60ddd7b3fb12 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:00.566900 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99290c18-cb62-45e0-9f10-60ddd7b3fb12 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:00.567064 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99290c18-cb62-45e0-9f10-60ddd7b3fb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:00.587570 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-99290c18-cb62-45e0-9f10-60ddd7b3fb12 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:20:00.588772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-99290c18-cb62-45e0-9f10-60ddd7b3fb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:20:00.658564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 559/1132] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:20:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:20:00.927765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90c054-2835-43e9-bf43-878c4ffed54c None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:00.962597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:01.036819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90c054-2835-43e9-bf43-878c4ffed54c None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:01.071562 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd56058-3fda-4804-bec7-a3c8ef63cf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:01.072176 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd56058-3fda-4804-bec7-a3c8ef63cf0f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:01.072450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd56058-3fda-4804-bec7-a3c8ef63cf0f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:01.073173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd56058-3fda-4804-bec7-a3c8ef63cf0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:01.074348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd56058-3fda-4804-bec7-a3c8ef63cf0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:01.075820 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd56058-3fda-4804-bec7-a3c8ef63cf0f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:01.077381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 574/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:20:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:01.083921 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99290c18-cb62-45e0-9f10-60ddd7b3fb12 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:01.129781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:01.198512 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99290c18-cb62-45e0-9f10-60ddd7b3fb12 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:01.234034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb35d1-56f2-4369-8eaf-22e9c64a33a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:01.234662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb35d1-56f2-4369-8eaf-22e9c64a33a7 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:01.234939 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb35d1-56f2-4369-8eaf-22e9c64a33a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:01.235201 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb35d1-56f2-4369-8eaf-22e9c64a33a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:01.235844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb35d1-56f2-4369-8eaf-22e9c64a33a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:01.237268 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb35d1-56f2-4369-8eaf-22e9c64a33a7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:01.269427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 560/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Sep 9 16:20:01 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:20:03.400362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653dc055-6b0a-4d69-8c32-4f2e38a354ef None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:03.401284 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653dc055-6b0a-4d69-8c32-4f2e38a354ef None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:03.401284 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653dc055-6b0a-4d69-8c32-4f2e38a354ef None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:03.403443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 575/1135] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:20:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:20:03.412597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093f667-7a16-48b6-b2a4-4c810e14cc41 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:03.413018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093f667-7a16-48b6-b2a4-4c810e14cc41 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:03.413159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093f667-7a16-48b6-b2a4-4c810e14cc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:03.443003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1093f667-7a16-48b6-b2a4-4c810e14cc41 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:20:03.444178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1093f667-7a16-48b6-b2a4-4c810e14cc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:20:03.517386 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 561/1136] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:20:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:20:03.793731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653dc055-6b0a-4d69-8c32-4f2e38a354ef None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:03.899523 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:03.911656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653dc055-6b0a-4d69-8c32-4f2e38a354ef None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:03.954821 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c38ab0-134d-4888-aaa7-388e0b69098f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:03.955441 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c38ab0-134d-4888-aaa7-388e0b69098f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:03.955717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c38ab0-134d-4888-aaa7-388e0b69098f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:03.955992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c38ab0-134d-4888-aaa7-388e0b69098f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:03.957170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c38ab0-134d-4888-aaa7-388e0b69098f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:03.958637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c38ab0-134d-4888-aaa7-388e0b69098f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:03.960224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 576/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:20:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:03.967053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093f667-7a16-48b6-b2a4-4c810e14cc41 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:03.994793 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:04.071911 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093f667-7a16-48b6-b2a4-4c810e14cc41 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:04.108103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90e0f7-dbdf-4d4c-a812-f57d543e941d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:04.108720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90e0f7-dbdf-4d4c-a812-f57d543e941d None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:04.108994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90e0f7-dbdf-4d4c-a812-f57d543e941d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:04.109263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90e0f7-dbdf-4d4c-a812-f57d543e941d None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:04.109904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90e0f7-dbdf-4d4c-a812-f57d543e941d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:04.111431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90e0f7-dbdf-4d4c-a812-f57d543e941d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:04.134857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 562/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:20:03 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:20:06.274757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3f443-6c7b-4856-8efb-a4cc6a3f44ff None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:06.275660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3f443-6c7b-4856-8efb-a4cc6a3f44ff None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:06.275660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3f443-6c7b-4856-8efb-a4cc6a3f44ff None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:06.277812 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 577/1139] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:20:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:20:06.284159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb7780-0e62-415e-9328-270b0ccdd31f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:06.284580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb7780-0e62-415e-9328-270b0ccdd31f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:06.284720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb7780-0e62-415e-9328-270b0ccdd31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:06.304114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ccbb7780-0e62-415e-9328-270b0ccdd31f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:20:06.305315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ccbb7780-0e62-415e-9328-270b0ccdd31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:20:06.379657 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 563/1140] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:20:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:20:06.658939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3f443-6c7b-4856-8efb-a4cc6a3f44ff None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:06.769529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3f443-6c7b-4856-8efb-a4cc6a3f44ff None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:06.803478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12b713-4210-4363-a9b0-2eaf730f31d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:06.804090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12b713-4210-4363-a9b0-2eaf730f31d5 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:06.804366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12b713-4210-4363-a9b0-2eaf730f31d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:06.804637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12b713-4210-4363-a9b0-2eaf730f31d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:06.815452 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12b713-4210-4363-a9b0-2eaf730f31d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-ea12b713-4210-4363-a9b0-2eaf730f31d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:20:06.820269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:20:06.821281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:20:06.822235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:20:06.822235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:20:06.822235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:20:06.822235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:20:06.822235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:20:06.822235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:20:06.822235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:20:06.822235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:20:06.822235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:20:06.822235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:20:06.822235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:20:06.822235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:20:06.822235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 578/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:20:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:20:06.828684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb7780-0e62-415e-9328-270b0ccdd31f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:06.840833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:06.932139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb7780-0e62-415e-9328-270b0ccdd31f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:06.968125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a127fc6-7624-4c59-b93e-dbb294919cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:06.968720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a127fc6-7624-4c59-b93e-dbb294919cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:06.968998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a127fc6-7624-4c59-b93e-dbb294919cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:06.969268 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a127fc6-7624-4c59-b93e-dbb294919cf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:06.969924 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a127fc6-7624-4c59-b93e-dbb294919cf9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:20:06.970228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a127fc6-7624-4c59-b93e-dbb294919cf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:06.971642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a127fc6-7624-4c59-b93e-dbb294919cf9 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:06.979779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 564/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:20:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:06.985913 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12b713-4210-4363-a9b0-2eaf730f31d5 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:07.012107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:07.090568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12b713-4210-4363-a9b0-2eaf730f31d5 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:07.128949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed631149-6d0c-473e-bf49-0dacd6e75a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:07.129606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed631149-6d0c-473e-bf49-0dacd6e75a43 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:07.129878 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed631149-6d0c-473e-bf49-0dacd6e75a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:07.130141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed631149-6d0c-473e-bf49-0dacd6e75a43 None admin] PATH_INFO: `/v3/users/10803ad4e4034d19a2ea6f464bb74679` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:07.144654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed631149-6d0c-473e-bf49-0dacd6e75a43 None admin] RBAC: Authorizing `identity:get_user(user_id=10803ad4e4034d19a2ea6f464bb74679)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:07.146124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed631149-6d0c-473e-bf49-0dacd6e75a43 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:07.147837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 579/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:20:06 2022] GET /identity/v3/users/10803ad4e4034d19a2ea6f464bb74679 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:07.154047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a127fc6-7624-4c59-b93e-dbb294919cf9 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:07.224641 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:07.255556 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a127fc6-7624-4c59-b93e-dbb294919cf9 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:07.291328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dde970-9940-4cff-9f88-132c947d733b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:07.291912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dde970-9940-4cff-9f88-132c947d733b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:07.292183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dde970-9940-4cff-9f88-132c947d733b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:07.292446 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dde970-9940-4cff-9f88-132c947d733b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:07.361039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dde970-9940-4cff-9f88-132c947d733b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:07.362477 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dde970-9940-4cff-9f88-132c947d733b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:07.436795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:07.477158 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 565/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:20:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:20:07.483355 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed631149-6d0c-473e-bf49-0dacd6e75a43 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:07.563714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:07.582818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed631149-6d0c-473e-bf49-0dacd6e75a43 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:07.617144 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fa481-5ad9-4034-9249-c2ef888d8b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:07.617757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fa481-5ad9-4034-9249-c2ef888d8b94 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:07.618030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fa481-5ad9-4034-9249-c2ef888d8b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:07.618299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fa481-5ad9-4034-9249-c2ef888d8b94 None admin] PATH_INFO: `/v3/projects/cb18290971c847a487a6e184e98f2ca5` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:07.619557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fa481-5ad9-4034-9249-c2ef888d8b94 None admin] RBAC: Authorizing `identity:get_project(project_id=cb18290971c847a487a6e184e98f2ca5)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:07.620983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fa481-5ad9-4034-9249-c2ef888d8b94 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:07.622594 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 580/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:20:07 2022] GET /identity/v3/projects/cb18290971c847a487a6e184e98f2ca5 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:07.629231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dde970-9940-4cff-9f88-132c947d733b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:07.731680 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dde970-9940-4cff-9f88-132c947d733b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:07.766576 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad3b75-a101-46da-8585-960a5d8fc312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:07.767166 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad3b75-a101-46da-8585-960a5d8fc312 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:07.767440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad3b75-a101-46da-8585-960a5d8fc312 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:07.767707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad3b75-a101-46da-8585-960a5d8fc312 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:07.768418 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad3b75-a101-46da-8585-960a5d8fc312 None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=cb18290971c847a487a6e184e98f2ca5, user.id=10803ad4e4034d19a2ea6f464bb74679) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:20:07.768726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad3b75-a101-46da-8585-960a5d8fc312 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:07.770144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad3b75-a101-46da-8585-960a5d8fc312 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:07.775593 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 566/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:20:07 2022] GET /identity/v3/role_assignments?user.id=10803ad4e4034d19a2ea6f464bb74679&scope.project.id=cb18290971c847a487a6e184e98f2ca5&role.id=c6da7ba2be504943af29338ee8855f05 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:09.859454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaa99b-d853-4652-bb98-b3c7d347c122 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:09.860023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaa99b-d853-4652-bb98-b3c7d347c122 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:09.860023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaa99b-d853-4652-bb98-b3c7d347c122 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:09.861322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 581/1147] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:20:09.867059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a1f19-a777-4573-afa1-cbf8789ca3ef None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:09.867470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a1f19-a777-4573-afa1-cbf8789ca3ef None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:09.867574 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a1f19-a777-4573-afa1-cbf8789ca3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:09.893210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-873a1f19-a777-4573-afa1-cbf8789ca3ef None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:20:09.894987 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-873a1f19-a777-4573-afa1-cbf8789ca3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:20:09.927349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:09.967828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 567/1148] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:20:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:20:10.244233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaa99b-d853-4652-bb98-b3c7d347c122 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:10.369049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaa99b-d853-4652-bb98-b3c7d347c122 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:10.410545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff480358-c428-49af-8189-d03c16c9c0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:10.411191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480358-c428-49af-8189-d03c16c9c0db None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:10.411466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480358-c428-49af-8189-d03c16c9c0db None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:10.411754 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480358-c428-49af-8189-d03c16c9c0db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:10.423449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff480358-c428-49af-8189-d03c16c9c0db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-ff480358-c428-49af-8189-d03c16c9c0db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:20:10.428327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:20:10.429480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:20:10.430652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:20:10.430652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:20:10.430652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:20:10.430652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:20:10.430652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:20:10.430652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:20:10.430652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:20:10.430652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:20:10.430652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:20:10.430652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:20:10.430652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:20:10.430652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:20:10.434443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 582/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:20:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:20:10.435675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a1f19-a777-4573-afa1-cbf8789ca3ef None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:10.536271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a1f19-a777-4573-afa1-cbf8789ca3ef None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:10.571852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c74e44-f74a-467a-a4b2-cd8ad59de3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:10.572464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c74e44-f74a-467a-a4b2-cd8ad59de3ba None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:10.572741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c74e44-f74a-467a-a4b2-cd8ad59de3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:10.573005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c74e44-f74a-467a-a4b2-cd8ad59de3ba None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:10.573684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c74e44-f74a-467a-a4b2-cd8ad59de3ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:20:10.573985 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c74e44-f74a-467a-a4b2-cd8ad59de3ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:10.575401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c74e44-f74a-467a-a4b2-cd8ad59de3ba None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:10.585180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 568/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:20:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:20:10.591469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff480358-c428-49af-8189-d03c16c9c0db None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:10.607764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:10.707374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff480358-c428-49af-8189-d03c16c9c0db None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:10.749213 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c8e66-f033-470d-992b-c4458069db38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:10.749840 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c8e66-f033-470d-992b-c4458069db38 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:10.750120 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c8e66-f033-470d-992b-c4458069db38 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:10.750388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c8e66-f033-470d-992b-c4458069db38 None admin] PATH_INFO: `/v3/users/10803ad4e4034d19a2ea6f464bb74679` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:10.751592 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c8e66-f033-470d-992b-c4458069db38 None admin] RBAC: Authorizing `identity:get_user(user_id=10803ad4e4034d19a2ea6f464bb74679)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:10.753039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c8e66-f033-470d-992b-c4458069db38 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:10.754653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 583/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:20:10 2022] GET /identity/v3/users/10803ad4e4034d19a2ea6f464bb74679 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:10.760957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c74e44-f74a-467a-a4b2-cd8ad59de3ba None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:10.790117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:10.873503 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c74e44-f74a-467a-a4b2-cd8ad59de3ba None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:10.911261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9d330-4d8a-4f56-bc16-8c678056bcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:10.911942 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9d330-4d8a-4f56-bc16-8c678056bcbf None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:10.912218 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9d330-4d8a-4f56-bc16-8c678056bcbf None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:10.912487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9d330-4d8a-4f56-bc16-8c678056bcbf None admin] PATH_INFO: `/v3/projects/cb18290971c847a487a6e184e98f2ca5` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:10.913848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9d330-4d8a-4f56-bc16-8c678056bcbf None admin] RBAC: Authorizing `identity:get_project(project_id=cb18290971c847a487a6e184e98f2ca5)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:10.915321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9d330-4d8a-4f56-bc16-8c678056bcbf None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:10.916968 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 569/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:20:10 2022] GET /identity/v3/projects/cb18290971c847a487a6e184e98f2ca5 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:10.923442 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c8e66-f033-470d-992b-c4458069db38 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:11.006386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:11.029035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c8e66-f033-470d-992b-c4458069db38 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:11.064052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28365b16-8b6d-4e5f-9973-e8edd1436e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:11.064644 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28365b16-8b6d-4e5f-9973-e8edd1436e13 None admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:11.064926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28365b16-8b6d-4e5f-9973-e8edd1436e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:11.065199 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28365b16-8b6d-4e5f-9973-e8edd1436e13 None admin] PATH_INFO: `/v3/projects/cb18290971c847a487a6e184e98f2ca5/users/10803ad4e4034d19a2ea6f464bb74679/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:11.067394 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28365b16-8b6d-4e5f-9973-e8edd1436e13 None admin] RBAC: Authorizing `identity:create_grant(project_id=cb18290971c847a487a6e184e98f2ca5, user_id=10803ad4e4034d19a2ea6f464bb74679, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:11.068919 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28365b16-8b6d-4e5f-9973-e8edd1436e13 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:11.086813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 584/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:20:10 2022] PUT /identity/v3/projects/cb18290971c847a487a6e184e98f2ca5/users/10803ad4e4034d19a2ea6f464bb74679/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:20:13.178436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07052d94-6bac-4c1b-83a6-5c888d51bea6 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:13.179022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07052d94-6bac-4c1b-83a6-5c888d51bea6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:13.179022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07052d94-6bac-4c1b-83a6-5c888d51bea6 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:13.181087 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 570/1154] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:20:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:20:13.189580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c3e82-102e-4707-93fc-4d672f57a110 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:13.190025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c3e82-102e-4707-93fc-4d672f57a110 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:13.190143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c3e82-102e-4707-93fc-4d672f57a110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:13.211567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-b26c3e82-102e-4707-93fc-4d672f57a110 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:20:13.212745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b26c3e82-102e-4707-93fc-4d672f57a110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:20:13.280476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 585/1155] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:20:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:20:13.551388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07052d94-6bac-4c1b-83a6-5c888d51bea6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:13.651022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:13.662685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07052d94-6bac-4c1b-83a6-5c888d51bea6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:13.703019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e073c4-ea6b-4cf4-8747-41c42c4d3f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:13.703641 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e073c4-ea6b-4cf4-8747-41c42c4d3f50 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:13.703919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e073c4-ea6b-4cf4-8747-41c42c4d3f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:13.704187 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e073c4-ea6b-4cf4-8747-41c42c4d3f50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:13.714523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e073c4-ea6b-4cf4-8747-41c42c4d3f50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-b3e073c4-ea6b-4cf4-8747-41c42c4d3f50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:20:13.719325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:20:13.720471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:20:13.721368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:20:13.721368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:20:13.721368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:20:13.721368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:20:13.721368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:20:13.721368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:20:13.721368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:20:13.721368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:20:13.721368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:20:13.721368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:20:13.721368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:20:13.721368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:20:13.721368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 571/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:20:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:20:13.727984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c3e82-102e-4707-93fc-4d672f57a110 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:13.807196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:13.841907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c3e82-102e-4707-93fc-4d672f57a110 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:13.876552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d41e2-8f8b-402e-950f-a8431f444eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:13.877151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d41e2-8f8b-402e-950f-a8431f444eec None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:13.877428 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d41e2-8f8b-402e-950f-a8431f444eec None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:13.877700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d41e2-8f8b-402e-950f-a8431f444eec None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:13.878353 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d41e2-8f8b-402e-950f-a8431f444eec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:20:13.878687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d41e2-8f8b-402e-950f-a8431f444eec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:13.880068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d41e2-8f8b-402e-950f-a8431f444eec None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:13.888064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 586/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:20:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:13.897211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e073c4-ea6b-4cf4-8747-41c42c4d3f50 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:14.004258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e073c4-ea6b-4cf4-8747-41c42c4d3f50 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:14.040169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68be15-0c1b-4124-a8ce-8d04365a3168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:14.040786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68be15-0c1b-4124-a8ce-8d04365a3168 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:14.041067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68be15-0c1b-4124-a8ce-8d04365a3168 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:14.041339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68be15-0c1b-4124-a8ce-8d04365a3168 None admin] PATH_INFO: `/v3/users/10803ad4e4034d19a2ea6f464bb74679` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:14.042606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68be15-0c1b-4124-a8ce-8d04365a3168 None admin] RBAC: Authorizing `identity:get_user(user_id=10803ad4e4034d19a2ea6f464bb74679)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:14.044038 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68be15-0c1b-4124-a8ce-8d04365a3168 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:14.045590 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 572/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:20:13 2022] GET /identity/v3/users/10803ad4e4034d19a2ea6f464bb74679 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:14.051638 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d41e2-8f8b-402e-950f-a8431f444eec None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:14.155062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d41e2-8f8b-402e-950f-a8431f444eec None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:14.183482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:14.191052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691585e3-67f9-4818-b96b-49acb0ee9f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:14.191639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691585e3-67f9-4818-b96b-49acb0ee9f09 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:14.191925 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691585e3-67f9-4818-b96b-49acb0ee9f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:14.192204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691585e3-67f9-4818-b96b-49acb0ee9f09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:14.258963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691585e3-67f9-4818-b96b-49acb0ee9f09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:14.260408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691585e3-67f9-4818-b96b-49acb0ee9f09 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:14.359909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 587/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:20:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:20:14.366930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68be15-0c1b-4124-a8ce-8d04365a3168 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:14.398726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:14.476249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68be15-0c1b-4124-a8ce-8d04365a3168 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:14.512076 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d023a30-6692-4e59-a09e-59e50d6e2a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:14.512693 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d023a30-6692-4e59-a09e-59e50d6e2a4a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:14.512969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d023a30-6692-4e59-a09e-59e50d6e2a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:14.513237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d023a30-6692-4e59-a09e-59e50d6e2a4a None admin] PATH_INFO: `/v3/projects/cb18290971c847a487a6e184e98f2ca5` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:14.514392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d023a30-6692-4e59-a09e-59e50d6e2a4a None admin] RBAC: Authorizing `identity:get_project(project_id=cb18290971c847a487a6e184e98f2ca5)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:14.515859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d023a30-6692-4e59-a09e-59e50d6e2a4a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:14.517481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 573/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:20:14 2022] GET /identity/v3/projects/cb18290971c847a487a6e184e98f2ca5 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:14.525190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691585e3-67f9-4818-b96b-49acb0ee9f09 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:14.555450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:14.628538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691585e3-67f9-4818-b96b-49acb0ee9f09 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:14.662888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb6723-5638-4988-a8a1-1ca5e97ed44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:14.663482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb6723-5638-4988-a8a1-1ca5e97ed44a None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:14.663758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb6723-5638-4988-a8a1-1ca5e97ed44a None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:14.664024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb6723-5638-4988-a8a1-1ca5e97ed44a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:14.664746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb6723-5638-4988-a8a1-1ca5e97ed44a None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=cb18290971c847a487a6e184e98f2ca5, user.id=10803ad4e4034d19a2ea6f464bb74679) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:20:14.665053 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb6723-5638-4988-a8a1-1ca5e97ed44a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:14.666477 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb6723-5638-4988-a8a1-1ca5e97ed44a None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:14.672432 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 588/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:20:14 2022] GET /identity/v3/role_assignments?user.id=10803ad4e4034d19a2ea6f464bb74679&scope.project.id=cb18290971c847a487a6e184e98f2ca5&role.id=c6da7ba2be504943af29338ee8855f05 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:16.818776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7a526-f9b3-4d94-bc17-2f8c84df7837 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:16.819673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7a526-f9b3-4d94-bc17-2f8c84df7837 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:16.819673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7a526-f9b3-4d94-bc17-2f8c84df7837 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:16.821789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 574/1162] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:20:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:20:16.830670 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16a20b-2b03-46bf-bcfe-8e22212f4e42 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:16.831095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16a20b-2b03-46bf-bcfe-8e22212f4e42 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:16.831227 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16a20b-2b03-46bf-bcfe-8e22212f4e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:16.852049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-6b16a20b-2b03-46bf-bcfe-8e22212f4e42 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:20:16.853473 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-6b16a20b-2b03-46bf-bcfe-8e22212f4e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:20:16.922045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 589/1163] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:20:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:20:17.186091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7a526-f9b3-4d94-bc17-2f8c84df7837 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:17.292252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:17.307739 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7a526-f9b3-4d94-bc17-2f8c84df7837 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:17.346728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c6bba-1a31-4755-97d6-9af6f2f11e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:17.347324 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c6bba-1a31-4755-97d6-9af6f2f11e3f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:17.347594 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c6bba-1a31-4755-97d6-9af6f2f11e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:17.347856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c6bba-1a31-4755-97d6-9af6f2f11e3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:17.419170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c6bba-1a31-4755-97d6-9af6f2f11e3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:17.420627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c6bba-1a31-4755-97d6-9af6f2f11e3f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:17.517422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:17.528812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 575/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:20:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:20:17.537775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16a20b-2b03-46bf-bcfe-8e22212f4e42 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:17.564457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:17.640114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16a20b-2b03-46bf-bcfe-8e22212f4e42 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:17.675160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4a30c-ab4f-410c-acfd-af7aaf8f0c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:17.675783 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4a30c-ab4f-410c-acfd-af7aaf8f0c9b None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:17.676059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4a30c-ab4f-410c-acfd-af7aaf8f0c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:17.676329 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4a30c-ab4f-410c-acfd-af7aaf8f0c9b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:17.683231 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab4a30c-ab4f-410c-acfd-af7aaf8f0c9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:17.684704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab4a30c-ab4f-410c-acfd-af7aaf8f0c9b None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-fab4a30c-ab4f-410c-acfd-af7aaf8f0c9b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:20:17.697399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:20:17.698684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:20:17.699687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 590/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Sep 9 16:20:17 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:20:17.705363 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c6bba-1a31-4755-97d6-9af6f2f11e3f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:17.814591 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c6bba-1a31-4755-97d6-9af6f2f11e3f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:17.852037 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4931a7-9657-4e02-8cd6-50b600af5b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:17.852669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4931a7-9657-4e02-8cd6-50b600af5b6d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:17.852897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4931a7-9657-4e02-8cd6-50b600af5b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:17.853162 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4931a7-9657-4e02-8cd6-50b600af5b6d None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:17.853792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4931a7-9657-4e02-8cd6-50b600af5b6d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:20:17.854087 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4931a7-9657-4e02-8cd6-50b600af5b6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:17.855476 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4931a7-9657-4e02-8cd6-50b600af5b6d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:17.862331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 576/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Sep 9 16:20:17 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:19.961079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b07d8-2b3a-4132-9d14-62075d2dcf43 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:19.961661 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b07d8-2b3a-4132-9d14-62075d2dcf43 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:19.961661 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b07d8-2b3a-4132-9d14-62075d2dcf43 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:19.963803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 591/1167] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:20:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:20:19.969342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b05f50-b689-446b-9419-9838956d17ec None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:19.969772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b05f50-b689-446b-9419-9838956d17ec None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:19.969881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b05f50-b689-446b-9419-9838956d17ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:19.996862 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c7b05f50-b689-446b-9419-9838956d17ec None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:20:19.998047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c7b05f50-b689-446b-9419-9838956d17ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:20:20.050619 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:20.068281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 577/1168] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:20:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:20:20.331112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b07d8-2b3a-4132-9d14-62075d2dcf43 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:20.433423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:20.451077 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b07d8-2b3a-4132-9d14-62075d2dcf43 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:20.484020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6f86f-169a-43d9-bc86-b3e04e8f063b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:20.484615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6f86f-169a-43d9-bc86-b3e04e8f063b None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:20.484893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6f86f-169a-43d9-bc86-b3e04e8f063b None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:20.485167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6f86f-169a-43d9-bc86-b3e04e8f063b None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:20.485819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6f86f-169a-43d9-bc86-b3e04e8f063b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:20.487224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6f86f-169a-43d9-bc86-b3e04e8f063b None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:20.505677 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 592/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Fri Sep 9 16:20:20 2022] POST /identity/v3/domains => generated 267 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:20:22.647735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029b42a-1cbc-45cc-84da-9049f2b8d871 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:22.648602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029b42a-1cbc-45cc-84da-9049f2b8d871 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:22.648602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029b42a-1cbc-45cc-84da-9049f2b8d871 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:22.650840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 578/1170] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:20:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:20:22.656255 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4eeeb-9482-43c5-980d-59e6fe37b03b None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:22.656697 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4eeeb-9482-43c5-980d-59e6fe37b03b None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:22.656788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4eeeb-9482-43c5-980d-59e6fe37b03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:22.676784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-a9e4eeeb-9482-43c5-980d-59e6fe37b03b None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:20:22.677954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a9e4eeeb-9482-43c5-980d-59e6fe37b03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:20:22.754974 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 593/1171] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:20:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:20:23.020032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029b42a-1cbc-45cc-84da-9049f2b8d871 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:23.139594 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029b42a-1cbc-45cc-84da-9049f2b8d871 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:23.176165 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1d7e6-07ff-4344-a810-5525c2885960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:23.176809 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1d7e6-07ff-4344-a810-5525c2885960 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:23.177077 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1d7e6-07ff-4344-a810-5525c2885960 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:23.177342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1d7e6-07ff-4344-a810-5525c2885960 None admin] PATH_INFO: `/v3/domains/de5f8e2452fe4f6ab2a8c39ae32ff141` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:23.188732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1d7e6-07ff-4344-a810-5525c2885960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=de5f8e2452fe4f6ab2a8c39ae32ff141)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:23.190178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1d7e6-07ff-4344-a810-5525c2885960 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:23.191785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 579/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Fri Sep 9 16:20:23 2022] GET /identity/v3/domains/de5f8e2452fe4f6ab2a8c39ae32ff141 => generated 267 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:23.198094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4eeeb-9482-43c5-980d-59e6fe37b03b None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:23.284962 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:23.313061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4eeeb-9482-43c5-980d-59e6fe37b03b None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:23.346341 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e92ff-ebb3-4aab-83a8-c46e762f89fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:23.346980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e92ff-ebb3-4aab-83a8-c46e762f89fd None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:23.347259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e92ff-ebb3-4aab-83a8-c46e762f89fd None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:23.347528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e92ff-ebb3-4aab-83a8-c46e762f89fd None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:23.348198 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e92ff-ebb3-4aab-83a8-c46e762f89fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:23.349777 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e92ff-ebb3-4aab-83a8-c46e762f89fd None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:23.384039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 594/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Sep 9 16:20:23 2022] POST /identity/v3/projects => generated 361 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:20:25.531702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c9d62-a891-486a-8129-b65d36b196ef None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:25.532580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c9d62-a891-486a-8129-b65d36b196ef None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:25.532580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c9d62-a891-486a-8129-b65d36b196ef None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:25.534711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 580/1174] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:20:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:20:25.540380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b18be-5eed-47b0-85a3-1638b88bbd31 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:25.540799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b18be-5eed-47b0-85a3-1638b88bbd31 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:25.540906 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b18be-5eed-47b0-85a3-1638b88bbd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:25.559914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-012b18be-5eed-47b0-85a3-1638b88bbd31 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:20:25.561078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-012b18be-5eed-47b0-85a3-1638b88bbd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:20:25.628902 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 595/1175] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:20:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:20:25.897096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c9d62-a891-486a-8129-b65d36b196ef None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:25.966947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:26.024134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c9d62-a891-486a-8129-b65d36b196ef None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:26.070431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37aa97-d9b3-44dc-838a-7a3cd14c4512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:26.071085 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37aa97-d9b3-44dc-838a-7a3cd14c4512 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:26.071360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37aa97-d9b3-44dc-838a-7a3cd14c4512 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:26.071629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37aa97-d9b3-44dc-838a-7a3cd14c4512 None admin] PATH_INFO: `/v3/domains/de5f8e2452fe4f6ab2a8c39ae32ff141` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:26.072809 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37aa97-d9b3-44dc-838a-7a3cd14c4512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=de5f8e2452fe4f6ab2a8c39ae32ff141)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:26.074255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37aa97-d9b3-44dc-838a-7a3cd14c4512 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:26.075892 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 581/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Fri Sep 9 16:20:25 2022] GET /identity/v3/domains/de5f8e2452fe4f6ab2a8c39ae32ff141 => generated 267 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:26.081789 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b18be-5eed-47b0-85a3-1638b88bbd31 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:26.138342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:26.188885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b18be-5eed-47b0-85a3-1638b88bbd31 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:26.223275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e243174-117a-435d-b88c-b8b430d2f942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:26.223928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e243174-117a-435d-b88c-b8b430d2f942 None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:26.224208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e243174-117a-435d-b88c-b8b430d2f942 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:26.224482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e243174-117a-435d-b88c-b8b430d2f942 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:26.225111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e243174-117a-435d-b88c-b8b430d2f942 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:26.226558 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e243174-117a-435d-b88c-b8b430d2f942 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:26.250992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 596/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:20:26 2022] POST /identity/v3/users => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:20:28.436785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9f031-830f-445b-b465-1ef84c198ca8 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:28.437689 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9f031-830f-445b-b465-1ef84c198ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:28.437689 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9f031-830f-445b-b465-1ef84c198ca8 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:28.440431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 582/1178] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:20:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:20:28.449552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014c47-12ab-4ff2-bf59-475ea7da5844 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:28.450224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014c47-12ab-4ff2-bf59-475ea7da5844 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:28.450409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014c47-12ab-4ff2-bf59-475ea7da5844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:28.477746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-0c014c47-12ab-4ff2-bf59-475ea7da5844 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:20:28.478990 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0c014c47-12ab-4ff2-bf59-475ea7da5844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:20:28.547814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 597/1179] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:20:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:20:28.815074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9f031-830f-445b-b465-1ef84c198ca8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:28.926422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9f031-830f-445b-b465-1ef84c198ca8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:28.941655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:28.963534 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b26a6-1c6e-4cb8-aee8-254123a7fbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:28.964154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b26a6-1c6e-4cb8-aee8-254123a7fbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:28.964429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b26a6-1c6e-4cb8-aee8-254123a7fbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:28.964698 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b26a6-1c6e-4cb8-aee8-254123a7fbc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:28.975363 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b26a6-1c6e-4cb8-aee8-254123a7fbc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-9d8b26a6-1c6e-4cb8-aee8-254123a7fbc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:20:28.980118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:20:28.981178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:20:28.982109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:20:28.982109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:20:28.982109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:20:28.982109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:20:28.982109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:20:28.982109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:20:28.982109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:20:28.982109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:20:28.982109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:20:28.982109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:20:28.982109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:20:28.982109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:20:28.982109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 583/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:20:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:20:28.987544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c014c47-12ab-4ff2-bf59-475ea7da5844 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:29.033744 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:29.105192 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c014c47-12ab-4ff2-bf59-475ea7da5844 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:29.145203 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361876da-571f-402f-ac7d-fda24756d9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:29.145812 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361876da-571f-402f-ac7d-fda24756d9ad None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:29.146088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361876da-571f-402f-ac7d-fda24756d9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:29.146354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361876da-571f-402f-ac7d-fda24756d9ad None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:29.147050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361876da-571f-402f-ac7d-fda24756d9ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:20:29.147351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361876da-571f-402f-ac7d-fda24756d9ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:29.148765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361876da-571f-402f-ac7d-fda24756d9ad None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:29.156869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 598/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:20:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:29.163816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b26a6-1c6e-4cb8-aee8-254123a7fbc2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:29.280666 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b26a6-1c6e-4cb8-aee8-254123a7fbc2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:29.317522 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e51088-b803-4249-b877-62199a24e91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:29.318135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e51088-b803-4249-b877-62199a24e91d None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:29.318412 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e51088-b803-4249-b877-62199a24e91d None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:29.318717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e51088-b803-4249-b877-62199a24e91d None admin] PATH_INFO: `/v3/users/8a402743ee844e98b4feedf29f46ccd6` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:29.332138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e51088-b803-4249-b877-62199a24e91d None admin] RBAC: Authorizing `identity:get_user(user_id=8a402743ee844e98b4feedf29f46ccd6)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:29.333671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e51088-b803-4249-b877-62199a24e91d None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:29.335328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 584/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:20:29 2022] GET /identity/v3/users/8a402743ee844e98b4feedf29f46ccd6 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:29.344549 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361876da-571f-402f-ac7d-fda24756d9ad None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:29.450286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361876da-571f-402f-ac7d-fda24756d9ad None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:29.452600 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:29.488510 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ed6b7-d360-4d6d-9f49-3f80d1e2b5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:29.489207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ed6b7-d360-4d6d-9f49-3f80d1e2b5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:29.489479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ed6b7-d360-4d6d-9f49-3f80d1e2b5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:29.489748 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ed6b7-d360-4d6d-9f49-3f80d1e2b5e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:29.556559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ed6b7-d360-4d6d-9f49-3f80d1e2b5e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:29.557985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ed6b7-d360-4d6d-9f49-3f80d1e2b5e8 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:29.655990 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 599/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:20:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:20:29.665752 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e51088-b803-4249-b877-62199a24e91d None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:29.702284 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:29.775556 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e51088-b803-4249-b877-62199a24e91d None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:29.811032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d78772-a11c-4f3f-9229-9a926d36b17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:29.811649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d78772-a11c-4f3f-9229-9a926d36b17f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:29.811925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d78772-a11c-4f3f-9229-9a926d36b17f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:29.812195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d78772-a11c-4f3f-9229-9a926d36b17f None admin] PATH_INFO: `/v3/projects/9d10dd916816466b87b61999d402da16` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:29.813719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d78772-a11c-4f3f-9229-9a926d36b17f None admin] RBAC: Authorizing `identity:get_project(project_id=9d10dd916816466b87b61999d402da16)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:29.815191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d78772-a11c-4f3f-9229-9a926d36b17f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:29.816802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 585/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:20:29 2022] GET /identity/v3/projects/9d10dd916816466b87b61999d402da16 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:29.823385 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ed6b7-d360-4d6d-9f49-3f80d1e2b5e8 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:29.826538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:29.925095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ed6b7-d360-4d6d-9f49-3f80d1e2b5e8 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:29.960094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f166b-6dbc-4f01-b3af-c0328d32d447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:29.960676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f166b-6dbc-4f01-b3af-c0328d32d447 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:29.960950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f166b-6dbc-4f01-b3af-c0328d32d447 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:29.961214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f166b-6dbc-4f01-b3af-c0328d32d447 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:29.961927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f166b-6dbc-4f01-b3af-c0328d32d447 None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=9d10dd916816466b87b61999d402da16, user.id=8a402743ee844e98b4feedf29f46ccd6) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:20:29.962227 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f166b-6dbc-4f01-b3af-c0328d32d447 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:29.963685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f166b-6dbc-4f01-b3af-c0328d32d447 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:29.969036 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 600/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:20:29 2022] GET /identity/v3/role_assignments?user.id=8a402743ee844e98b4feedf29f46ccd6&scope.project.id=9d10dd916816466b87b61999d402da16&role.id=c6da7ba2be504943af29338ee8855f05 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:32.052009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9464-dc01-4ef9-b9e9-d87dae423ada None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:32.052580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9464-dc01-4ef9-b9e9-d87dae423ada None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:32.052580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9464-dc01-4ef9-b9e9-d87dae423ada None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:32.053923 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 586/1186] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:20:32.060416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe071b9-8e40-48ff-a380-f659bff76ac3 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:32.061191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe071b9-8e40-48ff-a380-f659bff76ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:32.061352 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe071b9-8e40-48ff-a380-f659bff76ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:32.087663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-dfe071b9-8e40-48ff-a380-f659bff76ac3 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:20:32.088849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-dfe071b9-8e40-48ff-a380-f659bff76ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:20:32.151640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:32.157609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 601/1187] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:20:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:20:32.419598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed9464-dc01-4ef9-b9e9-d87dae423ada None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:32.533935 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:32.538347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed9464-dc01-4ef9-b9e9-d87dae423ada None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:32.576259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ff0f5-fc8d-4223-9ee2-9ef9c0ac8a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:32.576884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ff0f5-fc8d-4223-9ee2-9ef9c0ac8a8a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:32.577161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ff0f5-fc8d-4223-9ee2-9ef9c0ac8a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:32.577431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ff0f5-fc8d-4223-9ee2-9ef9c0ac8a8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:32.588118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ff0f5-fc8d-4223-9ee2-9ef9c0ac8a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-257ff0f5-fc8d-4223-9ee2-9ef9c0ac8a8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:20:32.592888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:20:32.593953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:20:32.595004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:20:32.595004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:20:32.595004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:20:32.595004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:20:32.595004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:20:32.595004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:20:32.595004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:20:32.595004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:20:32.595004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:20:32.595004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:20:32.595004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:20:32.595004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:20:32.595004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 587/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:20:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:20:32.600459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe071b9-8e40-48ff-a380-f659bff76ac3 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:32.708177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe071b9-8e40-48ff-a380-f659bff76ac3 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:32.742726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf05799-40de-49ed-9b33-66b319ea3f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:32.743332 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf05799-40de-49ed-9b33-66b319ea3f42 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:32.743605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf05799-40de-49ed-9b33-66b319ea3f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:32.743868 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf05799-40de-49ed-9b33-66b319ea3f42 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:32.744529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf05799-40de-49ed-9b33-66b319ea3f42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:20:32.744827 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf05799-40de-49ed-9b33-66b319ea3f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:32.746201 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf05799-40de-49ed-9b33-66b319ea3f42 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:32.754124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 602/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:20:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:32.760996 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ff0f5-fc8d-4223-9ee2-9ef9c0ac8a8a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:32.875972 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ff0f5-fc8d-4223-9ee2-9ef9c0ac8a8a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:32.910392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03af34-6cbe-4b1b-aecb-57914acf4074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:32.911127 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03af34-6cbe-4b1b-aecb-57914acf4074 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:32.911406 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03af34-6cbe-4b1b-aecb-57914acf4074 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:32.911676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03af34-6cbe-4b1b-aecb-57914acf4074 None admin] PATH_INFO: `/v3/users/8a402743ee844e98b4feedf29f46ccd6` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:32.912850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03af34-6cbe-4b1b-aecb-57914acf4074 None admin] RBAC: Authorizing `identity:get_user(user_id=8a402743ee844e98b4feedf29f46ccd6)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:32.914280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03af34-6cbe-4b1b-aecb-57914acf4074 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:32.915853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 588/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:20:32 2022] GET /identity/v3/users/8a402743ee844e98b4feedf29f46ccd6 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:32.921890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf05799-40de-49ed-9b33-66b319ea3f42 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:32.986477 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:33.039371 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf05799-40de-49ed-9b33-66b319ea3f42 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:33.077729 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9589b9-b828-470c-b2b5-b6e5b79d2617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:33.078324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9589b9-b828-470c-b2b5-b6e5b79d2617 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:33.078654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9589b9-b828-470c-b2b5-b6e5b79d2617 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:33.078925 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9589b9-b828-470c-b2b5-b6e5b79d2617 None admin] PATH_INFO: `/v3/projects/9d10dd916816466b87b61999d402da16` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:33.080102 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9589b9-b828-470c-b2b5-b6e5b79d2617 None admin] RBAC: Authorizing `identity:get_project(project_id=9d10dd916816466b87b61999d402da16)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:33.081540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9589b9-b828-470c-b2b5-b6e5b79d2617 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:33.083190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 603/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:20:32 2022] GET /identity/v3/projects/9d10dd916816466b87b61999d402da16 => generated 361 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:33.089473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03af34-6cbe-4b1b-aecb-57914acf4074 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:33.120356 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:33.194605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03af34-6cbe-4b1b-aecb-57914acf4074 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:33.229072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d0f33-a017-4086-ac12-b464ee1b7ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:33.229713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d0f33-a017-4086-ac12-b464ee1b7ebe None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:33.229983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d0f33-a017-4086-ac12-b464ee1b7ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:33.230245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d0f33-a017-4086-ac12-b464ee1b7ebe None admin] PATH_INFO: `/v3/projects/9d10dd916816466b87b61999d402da16/users/8a402743ee844e98b4feedf29f46ccd6/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:33.232390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d0f33-a017-4086-ac12-b464ee1b7ebe None admin] RBAC: Authorizing `identity:create_grant(project_id=9d10dd916816466b87b61999d402da16, user_id=8a402743ee844e98b4feedf29f46ccd6, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:33.233818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d0f33-a017-4086-ac12-b464ee1b7ebe None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:33.248448 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 589/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:20:33 2022] PUT /identity/v3/projects/9d10dd916816466b87b61999d402da16/users/8a402743ee844e98b4feedf29f46ccd6/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:20:35.363817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71157403-dacd-4eca-8513-cfa56fa61ef3 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:35.364684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71157403-dacd-4eca-8513-cfa56fa61ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:35.364684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71157403-dacd-4eca-8513-cfa56fa61ef3 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:35.366946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 604/1193] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:20:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:20:35.373333 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf419cc-e223-4096-ba7d-f203bb230da9 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:35.373772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf419cc-e223-4096-ba7d-f203bb230da9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:35.373889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf419cc-e223-4096-ba7d-f203bb230da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:35.396065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8cf419cc-e223-4096-ba7d-f203bb230da9 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:20:35.397211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8cf419cc-e223-4096-ba7d-f203bb230da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:20:35.465146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 590/1194] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:20:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:20:35.726764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71157403-dacd-4eca-8513-cfa56fa61ef3 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:35.836257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71157403-dacd-4eca-8513-cfa56fa61ef3 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:35.863807 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:35.874382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de476a-3ccd-42ee-8e87-b02d8159587c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:35.875041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de476a-3ccd-42ee-8e87-b02d8159587c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:35.875317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de476a-3ccd-42ee-8e87-b02d8159587c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:35.875582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de476a-3ccd-42ee-8e87-b02d8159587c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:35.887300 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de476a-3ccd-42ee-8e87-b02d8159587c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-f4de476a-3ccd-42ee-8e87-b02d8159587c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:20:35.892046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:20:35.893174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:20:35.894178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:20:35.894178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:20:35.894178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:20:35.894178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:20:35.894178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:20:35.894178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:20:35.894178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:20:35.894178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:20:35.894178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:20:35.894178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:20:35.894178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:20:35.894178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:20:35.894178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 605/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:20:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:20:35.900963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf419cc-e223-4096-ba7d-f203bb230da9 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:35.920447 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:36.003199 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf419cc-e223-4096-ba7d-f203bb230da9 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:36.047271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14755897-6134-471e-b8a7-94c22bcd9dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:36.047866 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14755897-6134-471e-b8a7-94c22bcd9dfa None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:36.048141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14755897-6134-471e-b8a7-94c22bcd9dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:36.048407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14755897-6134-471e-b8a7-94c22bcd9dfa None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:36.049062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14755897-6134-471e-b8a7-94c22bcd9dfa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:20:36.049357 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14755897-6134-471e-b8a7-94c22bcd9dfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:36.050752 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14755897-6134-471e-b8a7-94c22bcd9dfa None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:36.058849 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 591/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:20:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:36.065228 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de476a-3ccd-42ee-8e87-b02d8159587c None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:36.169193 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de476a-3ccd-42ee-8e87-b02d8159587c None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:36.204286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1f641-caa8-426f-93ec-81e5b3739049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:36.204911 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1f641-caa8-426f-93ec-81e5b3739049 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:36.205192 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1f641-caa8-426f-93ec-81e5b3739049 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:36.205465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1f641-caa8-426f-93ec-81e5b3739049 None admin] PATH_INFO: `/v3/users/8a402743ee844e98b4feedf29f46ccd6` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:36.206763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1f641-caa8-426f-93ec-81e5b3739049 None admin] RBAC: Authorizing `identity:get_user(user_id=8a402743ee844e98b4feedf29f46ccd6)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:36.208229 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1f641-caa8-426f-93ec-81e5b3739049 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:36.209818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 606/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Sep 9 16:20:36 2022] GET /identity/v3/users/8a402743ee844e98b4feedf29f46ccd6 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:36.216674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14755897-6134-471e-b8a7-94c22bcd9dfa None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:36.299231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:36.320121 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14755897-6134-471e-b8a7-94c22bcd9dfa None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:36.353253 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f79e60-cbdd-4192-b0f9-f712077b0b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:36.353851 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f79e60-cbdd-4192-b0f9-f712077b0b12 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:36.354130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f79e60-cbdd-4192-b0f9-f712077b0b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:36.354404 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f79e60-cbdd-4192-b0f9-f712077b0b12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:36.423332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f79e60-cbdd-4192-b0f9-f712077b0b12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:36.424782 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f79e60-cbdd-4192-b0f9-f712077b0b12 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:36.506969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:36.528376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 592/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:20:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:20:36.534593 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1f641-caa8-426f-93ec-81e5b3739049 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:36.594357 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:36.641522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1f641-caa8-426f-93ec-81e5b3739049 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:36.675240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239b3f3-cd16-42de-99bb-73ecc20740f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:36.675854 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239b3f3-cd16-42de-99bb-73ecc20740f6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:36.676126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239b3f3-cd16-42de-99bb-73ecc20740f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:36.676428 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239b3f3-cd16-42de-99bb-73ecc20740f6 None admin] PATH_INFO: `/v3/projects/9d10dd916816466b87b61999d402da16` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:36.677655 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239b3f3-cd16-42de-99bb-73ecc20740f6 None admin] RBAC: Authorizing `identity:get_project(project_id=9d10dd916816466b87b61999d402da16)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:36.679111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239b3f3-cd16-42de-99bb-73ecc20740f6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:36.680713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 607/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Sep 9 16:20:36 2022] GET /identity/v3/projects/9d10dd916816466b87b61999d402da16 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:36.690240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f79e60-cbdd-4192-b0f9-f712077b0b12 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:20:36.805627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f79e60-cbdd-4192-b0f9-f712077b0b12 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:20:36.845032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce6e0b-09bf-4f7d-a25e-9b2073288bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:20:36.845823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce6e0b-09bf-4f7d-a25e-9b2073288bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:20:36.846208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce6e0b-09bf-4f7d-a25e-9b2073288bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:20:36.846614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce6e0b-09bf-4f7d-a25e-9b2073288bb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:20:36.847626 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce6e0b-09bf-4f7d-a25e-9b2073288bb8 None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=9d10dd916816466b87b61999d402da16, user.id=8a402743ee844e98b4feedf29f46ccd6) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:20:36.848049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce6e0b-09bf-4f7d-a25e-9b2073288bb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:20:36.849548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce6e0b-09bf-4f7d-a25e-9b2073288bb8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:20:36.855815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 593/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:20:36 2022] GET /identity/v3/role_assignments?user.id=8a402743ee844e98b4feedf29f46ccd6&scope.project.id=9d10dd916816466b87b61999d402da16&role.id=c6da7ba2be504943af29338ee8855f05 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:20:46.670391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=38 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:20:46.852882 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=78 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:32.420441 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d992e0-c964-404d-8f86-d1d333659a67 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:32.421091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d992e0-c964-404d-8f86-d1d333659a67 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:32.421091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d992e0-c964-404d-8f86-d1d333659a67 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:32.422535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 608/1201] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:21:32.428836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa18458-bb85-4a3d-be3e-36efd9aa1481 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:32.429305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa18458-bb85-4a3d-be3e-36efd9aa1481 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:32.429393 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa18458-bb85-4a3d-be3e-36efd9aa1481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:32.452408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1fa18458-bb85-4a3d-be3e-36efd9aa1481 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:21:32.453644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1fa18458-bb85-4a3d-be3e-36efd9aa1481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:21:32.532444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 594/1202] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:21:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:21:32.797877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d992e0-c964-404d-8f86-d1d333659a67 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:32.919401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d992e0-c964-404d-8f86-d1d333659a67 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:32.972386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da53814-cd15-48ae-afc2-1ea78f017058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:32.973467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da53814-cd15-48ae-afc2-1ea78f017058 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:32.973952 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da53814-cd15-48ae-afc2-1ea78f017058 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:32.974421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da53814-cd15-48ae-afc2-1ea78f017058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:32.986802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da53814-cd15-48ae-afc2-1ea78f017058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:32.988431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da53814-cd15-48ae-afc2-1ea78f017058 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-0da53814-cd15-48ae-afc2-1ea78f017058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:21:33.005488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:33.006923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:21:33.007901 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:21:33.007901 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:33.007901 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:21:33.015764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 609/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:33.015954 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa18458-bb85-4a3d-be3e-36efd9aa1481 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:33.146699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa18458-bb85-4a3d-be3e-36efd9aa1481 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:33.190648 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:33.198322 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40b4d9-a5b2-43e4-b319-699ba5229f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:33.198994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40b4d9-a5b2-43e4-b319-699ba5229f5c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:33.199271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40b4d9-a5b2-43e4-b319-699ba5229f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:33.199541 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40b4d9-a5b2-43e4-b319-699ba5229f5c None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:33.200196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40b4d9-a5b2-43e4-b319-699ba5229f5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:33.200498 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40b4d9-a5b2-43e4-b319-699ba5229f5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:33.201885 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40b4d9-a5b2-43e4-b319-699ba5229f5c None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:33.215000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 595/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:33.220543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da53814-cd15-48ae-afc2-1ea78f017058 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:33.284983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:33.338594 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da53814-cd15-48ae-afc2-1ea78f017058 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:33.380173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c19ef-a494-43e0-82ac-74d32c3bb6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:33.380915 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c19ef-a494-43e0-82ac-74d32c3bb6f2 None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:33.381295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c19ef-a494-43e0-82ac-74d32c3bb6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:33.381646 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c19ef-a494-43e0-82ac-74d32c3bb6f2 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:33.382443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c19ef-a494-43e0-82ac-74d32c3bb6f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:33.384126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c19ef-a494-43e0-82ac-74d32c3bb6f2 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:33.418547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 610/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Sep 9 16:21:33 2022] POST /identity/v3/users => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:21:35.831426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba959f-aefd-49d8-920f-c074c3d56f5a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:35.832000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba959f-aefd-49d8-920f-c074c3d56f5a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:35.832000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba959f-aefd-49d8-920f-c074c3d56f5a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:35.833303 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 596/1206] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:21:35.839888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a2fe4-23ed-43a9-9b0c-95336491f840 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:35.840334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a2fe4-23ed-43a9-9b0c-95336491f840 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:35.840443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a2fe4-23ed-43a9-9b0c-95336491f840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:35.861140 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-066a2fe4-23ed-43a9-9b0c-95336491f840 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:21:35.862568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-066a2fe4-23ed-43a9-9b0c-95336491f840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:21:35.944726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 611/1207] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:21:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:21:36.268956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba959f-aefd-49d8-920f-c074c3d56f5a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:36.396626 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba959f-aefd-49d8-920f-c074c3d56f5a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:36.444468 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c58f6e-3332-4947-97b6-5fa2ee2478b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:36.445822 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c58f6e-3332-4947-97b6-5fa2ee2478b0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:36.446109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c58f6e-3332-4947-97b6-5fa2ee2478b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:36.446383 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c58f6e-3332-4947-97b6-5fa2ee2478b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:36.463560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c58f6e-3332-4947-97b6-5fa2ee2478b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-22c58f6e-3332-4947-97b6-5fa2ee2478b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:21:36.469609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:36.470673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:36.472032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:36.472032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:36.472032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:36.472032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:36.472032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:21:36.472032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:21:36.472032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:21:36.472032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:21:36.472032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:21:36.472032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:21:36.472032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:21:36.472032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:21:36.472032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 597/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:21:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:36.476868 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a2fe4-23ed-43a9-9b0c-95336491f840 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:36.519028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:36.584241 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a2fe4-23ed-43a9-9b0c-95336491f840 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:36.629319 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f169e-dd85-47dd-8805-9413bb51ef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:36.629986 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f169e-dd85-47dd-8805-9413bb51ef7d None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:36.630263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f169e-dd85-47dd-8805-9413bb51ef7d None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:36.630598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f169e-dd85-47dd-8805-9413bb51ef7d None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:36.631438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f169e-dd85-47dd-8805-9413bb51ef7d None admin] RBAC: Adding query filter params (name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:36.631779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f169e-dd85-47dd-8805-9413bb51ef7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:36.633495 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f169e-dd85-47dd-8805-9413bb51ef7d None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:36.647011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 612/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:21:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:36.649458 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c58f6e-3332-4947-97b6-5fa2ee2478b0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:36.708335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:36.769852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c58f6e-3332-4947-97b6-5fa2ee2478b0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:36.813653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68daa75-9c0b-4144-ae60-53d3cc9ff823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:36.814342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68daa75-9c0b-4144-ae60-53d3cc9ff823 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:36.814654 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68daa75-9c0b-4144-ae60-53d3cc9ff823 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:36.814955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68daa75-9c0b-4144-ae60-53d3cc9ff823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:36.828990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68daa75-9c0b-4144-ae60-53d3cc9ff823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:36.830864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68daa75-9c0b-4144-ae60-53d3cc9ff823 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-d68daa75-9c0b-4144-ae60-53d3cc9ff823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:21:36.848974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:36.855580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:21:36.856707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:21:36.856707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:36.856707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:21:36.856707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 598/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:36.864067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f169e-dd85-47dd-8805-9413bb51ef7d None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:36.971829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:36.975429 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f169e-dd85-47dd-8805-9413bb51ef7d None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:37.017113 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe89a9c-4720-407b-b864-e7b4fc1d4e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:37.017747 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe89a9c-4720-407b-b864-e7b4fc1d4e07 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:37.018173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe89a9c-4720-407b-b864-e7b4fc1d4e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:37.018377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe89a9c-4720-407b-b864-e7b4fc1d4e07 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:37.019205 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe89a9c-4720-407b-b864-e7b4fc1d4e07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:37.019506 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe89a9c-4720-407b-b864-e7b4fc1d4e07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:37.020963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe89a9c-4720-407b-b864-e7b4fc1d4e07 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:37.033967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 613/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:37.040179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68daa75-9c0b-4144-ae60-53d3cc9ff823 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:37.153744 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68daa75-9c0b-4144-ae60-53d3cc9ff823 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:37.155655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:37.193267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab3d34-2a2e-4005-997a-e3d36bbc13b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:37.193896 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3d34-2a2e-4005-997a-e3d36bbc13b7 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:37.194182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3d34-2a2e-4005-997a-e3d36bbc13b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:37.194451 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3d34-2a2e-4005-997a-e3d36bbc13b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:37.202080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab3d34-2a2e-4005-997a-e3d36bbc13b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:37.203550 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab3d34-2a2e-4005-997a-e3d36bbc13b7 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-4eab3d34-2a2e-4005-997a-e3d36bbc13b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:21:37.214906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:37.216026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:37.217009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:37.217009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:37.217009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:21:37.217009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:21:37.217009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:21:37.217009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:21:37.217009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:21:37.217009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:21:37.217009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 16:21:37.217009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:21:37.217009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 599/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:37.222544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe89a9c-4720-407b-b864-e7b4fc1d4e07 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:37.338006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe89a9c-4720-407b-b864-e7b4fc1d4e07 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:37.380561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f24e8e-04aa-4586-9968-93de1abbae93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:37.381176 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f24e8e-04aa-4586-9968-93de1abbae93 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:37.381465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f24e8e-04aa-4586-9968-93de1abbae93 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:37.381778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f24e8e-04aa-4586-9968-93de1abbae93 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:37.382519 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f24e8e-04aa-4586-9968-93de1abbae93 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:37.382823 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f24e8e-04aa-4586-9968-93de1abbae93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:37.384245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f24e8e-04aa-4586-9968-93de1abbae93 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:37.402797 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 614/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Sep 9 16:21:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:37.408584 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab3d34-2a2e-4005-997a-e3d36bbc13b7 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:37.528539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab3d34-2a2e-4005-997a-e3d36bbc13b7 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:37.567873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da838d4-d43a-4383-bebc-77fa35a0f9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:37.568493 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da838d4-d43a-4383-bebc-77fa35a0f9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:37.568769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da838d4-d43a-4383-bebc-77fa35a0f9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:37.569036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da838d4-d43a-4383-bebc-77fa35a0f9b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:37.599831 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:37.639071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da838d4-d43a-4383-bebc-77fa35a0f9b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:37.640560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da838d4-d43a-4383-bebc-77fa35a0f9b5 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:37.741052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 600/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:21:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:21:37.747137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f24e8e-04aa-4586-9968-93de1abbae93 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:37.766656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:37.862044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f24e8e-04aa-4586-9968-93de1abbae93 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:37.901704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e8aa8-c759-4509-b17d-8f7fda3787cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:37.902395 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8aa8-c759-4509-b17d-8f7fda3787cc None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:37.902616 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8aa8-c759-4509-b17d-8f7fda3787cc None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:37.902890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8aa8-c759-4509-b17d-8f7fda3787cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:37.913530 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e8aa8-c759-4509-b17d-8f7fda3787cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:37.915018 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e8aa8-c759-4509-b17d-8f7fda3787cc None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-041e8aa8-c759-4509-b17d-8f7fda3787cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:21:37.929772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:37.930819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:21:37.937403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:21:37.937403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:37.937403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:21:37.937403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 615/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:37.939613 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da838d4-d43a-4383-bebc-77fa35a0f9b5 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:38.004019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:38.051215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da838d4-d43a-4383-bebc-77fa35a0f9b5 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:38.106758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46289a28-b9de-46cf-b1d2-2ddbb92ea72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:38.107379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46289a28-b9de-46cf-b1d2-2ddbb92ea72b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:38.107652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46289a28-b9de-46cf-b1d2-2ddbb92ea72b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:38.107919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46289a28-b9de-46cf-b1d2-2ddbb92ea72b None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:38.108985 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46289a28-b9de-46cf-b1d2-2ddbb92ea72b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:38.108985 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46289a28-b9de-46cf-b1d2-2ddbb92ea72b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:38.110295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46289a28-b9de-46cf-b1d2-2ddbb92ea72b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:38.125981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 601/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:38.132216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e8aa8-c759-4509-b17d-8f7fda3787cc None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:38.202989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:38.237037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e8aa8-c759-4509-b17d-8f7fda3787cc None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:38.275996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c342ae-8433-4a96-9985-30d8be134937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:38.276690 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c342ae-8433-4a96-9985-30d8be134937 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:38.277007 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c342ae-8433-4a96-9985-30d8be134937 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:38.277318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c342ae-8433-4a96-9985-30d8be134937 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:38.285067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c342ae-8433-4a96-9985-30d8be134937 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:38.286568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c342ae-8433-4a96-9985-30d8be134937 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-77c342ae-8433-4a96-9985-30d8be134937 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:21:38.298855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:38.299993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:38.301003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:21:38.301003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:21:38.301003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:21:38.301003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:21:38.301003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:21:38.301003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:21:38.301003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:21:38.301003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:21:38.301003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 616/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:21:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:38.306832 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46289a28-b9de-46cf-b1d2-2ddbb92ea72b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:38.424275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46289a28-b9de-46cf-b1d2-2ddbb92ea72b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:38.449524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:38.462286 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b3634-f582-44aa-81fc-317db38120fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:38.462935 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b3634-f582-44aa-81fc-317db38120fb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:38.463214 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b3634-f582-44aa-81fc-317db38120fb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:38.463480 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b3634-f582-44aa-81fc-317db38120fb None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:38.464174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b3634-f582-44aa-81fc-317db38120fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:38.464474 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b3634-f582-44aa-81fc-317db38120fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:38.465927 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b3634-f582-44aa-81fc-317db38120fb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:38.478792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 602/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:21:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:38.485482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c342ae-8433-4a96-9985-30d8be134937 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:38.589525 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c342ae-8433-4a96-9985-30d8be134937 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:38.619146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:38.625468 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6b547-e023-4fdc-ba41-436b53d3e6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:38.626168 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6b547-e023-4fdc-ba41-436b53d3e6ae None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:38.626424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6b547-e023-4fdc-ba41-436b53d3e6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:38.626720 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6b547-e023-4fdc-ba41-436b53d3e6ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:38.627468 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6b547-e023-4fdc-ba41-436b53d3e6ae None admin] RBAC: Adding query filter params (role.id=2b68b618a1b846f69bbe9aa08852a66f, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=784b6147de91435292e4cf3940bcad9c) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:38.627788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6b547-e023-4fdc-ba41-436b53d3e6ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:38.629370 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6b547-e023-4fdc-ba41-436b53d3e6ae None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:38.635010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 617/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:21:38 2022] GET /identity/v3/role_assignments?user.id=784b6147de91435292e4cf3940bcad9c&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=2b68b618a1b846f69bbe9aa08852a66f => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:40.815893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f3987-4541-45ae-8aac-7ada65cb8624 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:40.816516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f3987-4541-45ae-8aac-7ada65cb8624 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:40.816516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f3987-4541-45ae-8aac-7ada65cb8624 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:40.817932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 603/1220] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:21:40.823805 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084a03c-26b5-4a58-9ed7-4efabd6e89ac None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:40.824586 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084a03c-26b5-4a58-9ed7-4efabd6e89ac None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:40.824840 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084a03c-26b5-4a58-9ed7-4efabd6e89ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:40.843694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-b084a03c-26b5-4a58-9ed7-4efabd6e89ac None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:21:40.844877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b084a03c-26b5-4a58-9ed7-4efabd6e89ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:21:40.909554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 618/1221] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:21:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:21:41.185726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f3987-4541-45ae-8aac-7ada65cb8624 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:41.296642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f3987-4541-45ae-8aac-7ada65cb8624 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:41.332494 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94befc81-68cf-4862-8e58-dd272e95f6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:41.333156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94befc81-68cf-4862-8e58-dd272e95f6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:41.333432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94befc81-68cf-4862-8e58-dd272e95f6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:41.333754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94befc81-68cf-4862-8e58-dd272e95f6d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:41.343863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94befc81-68cf-4862-8e58-dd272e95f6d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-94befc81-68cf-4862-8e58-dd272e95f6d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:21:41.348660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:41.349600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:41.350469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:41.350469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:41.350469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:41.350469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:41.350469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:21:41.350469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:21:41.350469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:21:41.350469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:21:41.350469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:21:41.350469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:21:41.350469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:21:41.350469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:21:41.350469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 604/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:21:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:41.358051 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084a03c-26b5-4a58-9ed7-4efabd6e89ac None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:41.460782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084a03c-26b5-4a58-9ed7-4efabd6e89ac None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:41.485754 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:41.497677 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1ab95-9153-40f7-9385-95256ebf7a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:41.498288 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1ab95-9153-40f7-9385-95256ebf7a53 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:41.498597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1ab95-9153-40f7-9385-95256ebf7a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:41.498865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1ab95-9153-40f7-9385-95256ebf7a53 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:41.499566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1ab95-9153-40f7-9385-95256ebf7a53 None admin] RBAC: Adding query filter params (name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:41.499869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1ab95-9153-40f7-9385-95256ebf7a53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:41.501210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1ab95-9153-40f7-9385-95256ebf7a53 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:41.509376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 619/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:21:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:21:41.515496 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94befc81-68cf-4862-8e58-dd272e95f6d2 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:41.565810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:41.623183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94befc81-68cf-4862-8e58-dd272e95f6d2 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:41.660168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b9e8f-2262-4a9c-a409-c3b5e59b3c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:41.660900 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b9e8f-2262-4a9c-a409-c3b5e59b3c72 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:41.661249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b9e8f-2262-4a9c-a409-c3b5e59b3c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:41.661588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b9e8f-2262-4a9c-a409-c3b5e59b3c72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:41.671905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b9e8f-2262-4a9c-a409-c3b5e59b3c72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:41.673431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b9e8f-2262-4a9c-a409-c3b5e59b3c72 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-be4b9e8f-2262-4a9c-a409-c3b5e59b3c72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:21:41.688245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:41.689529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:21:41.690673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:21:41.690673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:41.690673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:21:41.690673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 605/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:21:41.695566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1ab95-9153-40f7-9385-95256ebf7a53 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:41.804320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1ab95-9153-40f7-9385-95256ebf7a53 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:41.838746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1fc96-eac6-4e97-837e-1be9168bb726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:41.839520 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1fc96-eac6-4e97-837e-1be9168bb726 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:41.839642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1fc96-eac6-4e97-837e-1be9168bb726 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:41.839880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1fc96-eac6-4e97-837e-1be9168bb726 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:41.840521 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1fc96-eac6-4e97-837e-1be9168bb726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:41.840830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1fc96-eac6-4e97-837e-1be9168bb726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:41.842223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1fc96-eac6-4e97-837e-1be9168bb726 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:41.856263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 620/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:41.862174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b9e8f-2262-4a9c-a409-c3b5e59b3c72 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:41.964997 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:41.972429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b9e8f-2262-4a9c-a409-c3b5e59b3c72 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:42.011131 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28462c-62ed-4056-b674-83868126a47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:42.011753 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28462c-62ed-4056-b674-83868126a47a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:42.012036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28462c-62ed-4056-b674-83868126a47a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:42.012300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28462c-62ed-4056-b674-83868126a47a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:42.020461 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28462c-62ed-4056-b674-83868126a47a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:42.022092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28462c-62ed-4056-b674-83868126a47a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-4d28462c-62ed-4056-b674-83868126a47a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:21:42.034962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:42.036082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:42.037095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:42.037095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:42.037095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:21:42.037095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:21:42.037095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:21:42.037095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:21:42.037095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:21:42.037095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:21:42.037095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 16:21:42.037095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:21:42.037095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 606/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:42.042893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1fc96-eac6-4e97-837e-1be9168bb726 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:42.100754 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:42.162486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1fc96-eac6-4e97-837e-1be9168bb726 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:42.202457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c083ac0-6527-4cdf-8066-2d1cbaff6547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:42.203262 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c083ac0-6527-4cdf-8066-2d1cbaff6547 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:42.203625 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c083ac0-6527-4cdf-8066-2d1cbaff6547 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:42.204000 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c083ac0-6527-4cdf-8066-2d1cbaff6547 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:42.204829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c083ac0-6527-4cdf-8066-2d1cbaff6547 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:42.205224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c083ac0-6527-4cdf-8066-2d1cbaff6547 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:42.206873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c083ac0-6527-4cdf-8066-2d1cbaff6547 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:42.226168 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 621/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Sep 9 16:21:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:42.232082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28462c-62ed-4056-b674-83868126a47a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:42.338676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28462c-62ed-4056-b674-83868126a47a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:42.375884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab2704-3f9b-4742-9b96-70210a6f04cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:42.376552 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab2704-3f9b-4742-9b96-70210a6f04cb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:42.376880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab2704-3f9b-4742-9b96-70210a6f04cb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:42.377171 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab2704-3f9b-4742-9b96-70210a6f04cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:42.389635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab2704-3f9b-4742-9b96-70210a6f04cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:42.391257 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab2704-3f9b-4742-9b96-70210a6f04cb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-6bab2704-3f9b-4742-9b96-70210a6f04cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:21:42.406044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:42.407260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:21:42.408293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:21:42.408293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:42.408293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:21:42.408293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 607/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:42.414018 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c083ac0-6527-4cdf-8066-2d1cbaff6547 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:42.536327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c083ac0-6527-4cdf-8066-2d1cbaff6547 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:42.538662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:42.571881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ea3fe-998c-4bd6-8eb0-402ae1282da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:42.572505 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ea3fe-998c-4bd6-8eb0-402ae1282da2 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:42.572788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ea3fe-998c-4bd6-8eb0-402ae1282da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:42.573116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ea3fe-998c-4bd6-8eb0-402ae1282da2 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:42.573772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ea3fe-998c-4bd6-8eb0-402ae1282da2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:42.574081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ea3fe-998c-4bd6-8eb0-402ae1282da2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:42.575628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ea3fe-998c-4bd6-8eb0-402ae1282da2 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:42.588940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 622/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:21:42.595084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab2704-3f9b-4742-9b96-70210a6f04cb None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:42.605805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:42.711214 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab2704-3f9b-4742-9b96-70210a6f04cb None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:42.747960 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea54a6-f252-4b29-816c-06fa7200fb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:42.748808 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea54a6-f252-4b29-816c-06fa7200fb20 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:42.749120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea54a6-f252-4b29-816c-06fa7200fb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:42.749417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea54a6-f252-4b29-816c-06fa7200fb20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:42.758016 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea54a6-f252-4b29-816c-06fa7200fb20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:42.759684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea54a6-f252-4b29-816c-06fa7200fb20 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-6cea54a6-f252-4b29-816c-06fa7200fb20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:21:42.772625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:42.773649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:42.774929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:21:42.774929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:21:42.774929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:21:42.774929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:21:42.774929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:21:42.774929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:21:42.774929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:21:42.774929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:21:42.774929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 608/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:21:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:42.780632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ea3fe-998c-4bd6-8eb0-402ae1282da2 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:42.899436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ea3fe-998c-4bd6-8eb0-402ae1282da2 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:42.936587 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f6a5a-eadb-4199-9c10-df67f320f4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:42.937250 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f6a5a-eadb-4199-9c10-df67f320f4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:42.937554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f6a5a-eadb-4199-9c10-df67f320f4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:42.937946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f6a5a-eadb-4199-9c10-df67f320f4a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:42.938727 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f6a5a-eadb-4199-9c10-df67f320f4a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:42.939113 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f6a5a-eadb-4199-9c10-df67f320f4a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:42.940583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f6a5a-eadb-4199-9c10-df67f320f4a5 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:42.954610 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 623/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:21:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:42.961103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea54a6-f252-4b29-816c-06fa7200fb20 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:43.042951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:43.077105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea54a6-f252-4b29-816c-06fa7200fb20 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:43.114009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9352129-6eda-4cbf-a003-6d96bed9e416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:43.114704 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9352129-6eda-4cbf-a003-6d96bed9e416 None admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:43.114983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9352129-6eda-4cbf-a003-6d96bed9e416 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:43.115304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9352129-6eda-4cbf-a003-6d96bed9e416 None admin] PATH_INFO: `/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/784b6147de91435292e4cf3940bcad9c/roles/2b68b618a1b846f69bbe9aa08852a66f` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:43.134138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9352129-6eda-4cbf-a003-6d96bed9e416 None admin] RBAC: Authorizing `identity:create_grant(project_id=253ff30a56444df28d44c4b7eaadb276, user_id=784b6147de91435292e4cf3940bcad9c, role_id=2b68b618a1b846f69bbe9aa08852a66f)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:43.135719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9352129-6eda-4cbf-a003-6d96bed9e416 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:43.149955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 609/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:21:42 2022] PUT /identity/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/784b6147de91435292e4cf3940bcad9c/roles/2b68b618a1b846f69bbe9aa08852a66f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:21:45.307306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996f72e-8f60-4019-b94e-90d331135ed5 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:45.307872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996f72e-8f60-4019-b94e-90d331135ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:45.307872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996f72e-8f60-4019-b94e-90d331135ed5 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:45.309191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 624/1233] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:21:45.315873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568b739-3655-47e2-ad52-155cb30e4245 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:45.316435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568b739-3655-47e2-ad52-155cb30e4245 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:45.316435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568b739-3655-47e2-ad52-155cb30e4245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:45.340481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d568b739-3655-47e2-ad52-155cb30e4245 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:21:45.341706 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d568b739-3655-47e2-ad52-155cb30e4245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:21:45.421639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 610/1234] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:21:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:21:45.687490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996f72e-8f60-4019-b94e-90d331135ed5 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:45.714208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:45.797865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996f72e-8f60-4019-b94e-90d331135ed5 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:45.835073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7248ef-b354-42a1-b583-661b5cd5067d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:45.835714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7248ef-b354-42a1-b583-661b5cd5067d None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:45.835985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7248ef-b354-42a1-b583-661b5cd5067d None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:45.836267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7248ef-b354-42a1-b583-661b5cd5067d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:45.847378 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7248ef-b354-42a1-b583-661b5cd5067d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-3f7248ef-b354-42a1-b583-661b5cd5067d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:21:45.853184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:45.854374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:45.855296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:45.855296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:45.855296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:45.855296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:45.855296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:21:45.855296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:21:45.855296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:21:45.855296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:21:45.855296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:21:45.855296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:21:45.855296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 16:21:45.855296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:21:45.855296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 625/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Sep 9 16:21:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:45.861491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568b739-3655-47e2-ad52-155cb30e4245 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:45.913056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:45.974774 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568b739-3655-47e2-ad52-155cb30e4245 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:46.011483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113f542-9cee-4e3f-a8ec-aa5fdc3b4a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:46.012095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113f542-9cee-4e3f-a8ec-aa5fdc3b4a24 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:46.012399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113f542-9cee-4e3f-a8ec-aa5fdc3b4a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:46.012713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113f542-9cee-4e3f-a8ec-aa5fdc3b4a24 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:46.013548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113f542-9cee-4e3f-a8ec-aa5fdc3b4a24 None admin] RBAC: Adding query filter params (name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:46.013869 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113f542-9cee-4e3f-a8ec-aa5fdc3b4a24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:46.015609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113f542-9cee-4e3f-a8ec-aa5fdc3b4a24 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:46.025198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 611/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Sep 9 16:21:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:46.032159 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7248ef-b354-42a1-b583-661b5cd5067d None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:46.129254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:46.146887 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7248ef-b354-42a1-b583-661b5cd5067d None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:46.183813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b4cdc-f2ed-4a7d-b468-1ccaa2edaec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:46.185108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b4cdc-f2ed-4a7d-b468-1ccaa2edaec8 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:46.185108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b4cdc-f2ed-4a7d-b468-1ccaa2edaec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:46.185325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b4cdc-f2ed-4a7d-b468-1ccaa2edaec8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:46.196291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b4cdc-f2ed-4a7d-b468-1ccaa2edaec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:46.197900 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b4cdc-f2ed-4a7d-b468-1ccaa2edaec8 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-df8b4cdc-f2ed-4a7d-b468-1ccaa2edaec8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:21:46.213577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:46.219721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:21:46.221106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:21:46.221106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:46.221106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:21:46.221106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 626/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:21:46.221511 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113f542-9cee-4e3f-a8ec-aa5fdc3b4a24 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:46.332328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113f542-9cee-4e3f-a8ec-aa5fdc3b4a24 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:46.341279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:46.369428 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb3bb1-6664-422d-96a2-81c8eed97c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:46.370038 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3bb1-6664-422d-96a2-81c8eed97c65 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:46.370313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3bb1-6664-422d-96a2-81c8eed97c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:46.370606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3bb1-6664-422d-96a2-81c8eed97c65 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:46.371247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb3bb1-6664-422d-96a2-81c8eed97c65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:46.371544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb3bb1-6664-422d-96a2-81c8eed97c65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:46.372906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb3bb1-6664-422d-96a2-81c8eed97c65 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:46.384572 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 612/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:46.391390 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b4cdc-f2ed-4a7d-b468-1ccaa2edaec8 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:46.501421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b4cdc-f2ed-4a7d-b468-1ccaa2edaec8 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:46.538050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff72cf1-3cba-4e22-bc1f-d9c23a6874b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:46.538742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff72cf1-3cba-4e22-bc1f-d9c23a6874b7 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:46.539032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff72cf1-3cba-4e22-bc1f-d9c23a6874b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:46.539342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff72cf1-3cba-4e22-bc1f-d9c23a6874b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:46.542163 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:46.547702 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff72cf1-3cba-4e22-bc1f-d9c23a6874b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:46.549626 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff72cf1-3cba-4e22-bc1f-d9c23a6874b7 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-9ff72cf1-3cba-4e22-bc1f-d9c23a6874b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:21:46.561766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:46.567907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:46.569109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:46.569109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:46.569109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:21:46.569109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:21:46.569109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:21:46.569109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:21:46.569109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:21:46.569109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:21:46.569109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 16:21:46.569109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:21:46.569109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 627/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:46.569988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb3bb1-6664-422d-96a2-81c8eed97c65 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:46.685408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb3bb1-6664-422d-96a2-81c8eed97c65 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:46.727868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83b604-b271-4925-8988-d82ed28547cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:46.728509 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83b604-b271-4925-8988-d82ed28547cb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:46.728788 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83b604-b271-4925-8988-d82ed28547cb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:46.729053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83b604-b271-4925-8988-d82ed28547cb None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:46.729781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83b604-b271-4925-8988-d82ed28547cb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:46.730098 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83b604-b271-4925-8988-d82ed28547cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:46.731568 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83b604-b271-4925-8988-d82ed28547cb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:46.751170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 613/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Sep 9 16:21:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:46.757447 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff72cf1-3cba-4e22-bc1f-d9c23a6874b7 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:46.868779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff72cf1-3cba-4e22-bc1f-d9c23a6874b7 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:46.907199 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a9758-1c74-4faf-8d77-7c65e757d961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:46.907897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a9758-1c74-4faf-8d77-7c65e757d961 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:46.908360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a9758-1c74-4faf-8d77-7c65e757d961 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:46.908707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a9758-1c74-4faf-8d77-7c65e757d961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:46.972859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:46.979635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a9758-1c74-4faf-8d77-7c65e757d961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:46.981072 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a9758-1c74-4faf-8d77-7c65e757d961 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:47.091572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 628/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:21:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:21:47.097807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83b604-b271-4925-8988-d82ed28547cb None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:47.128307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:47.208731 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83b604-b271-4925-8988-d82ed28547cb None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:47.246619 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e620f1-602a-4d1b-ba3b-a716feb8d28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:47.247470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e620f1-602a-4d1b-ba3b-a716feb8d28b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:47.247837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e620f1-602a-4d1b-ba3b-a716feb8d28b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:47.248106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e620f1-602a-4d1b-ba3b-a716feb8d28b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:47.258998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e620f1-602a-4d1b-ba3b-a716feb8d28b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:47.260632 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e620f1-602a-4d1b-ba3b-a716feb8d28b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-70e620f1-602a-4d1b-ba3b-a716feb8d28b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:21:47.277373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:47.278755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:21:47.279934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:21:47.279934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:47.279934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:21:47.284090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 614/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:47.285014 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a9758-1c74-4faf-8d77-7c65e757d961 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:47.391466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:47.398133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a9758-1c74-4faf-8d77-7c65e757d961 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:47.439467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96822f1e-6d07-460f-b853-3a14cc446409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:47.440178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96822f1e-6d07-460f-b853-3a14cc446409 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:47.440460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96822f1e-6d07-460f-b853-3a14cc446409 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:47.440732 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96822f1e-6d07-460f-b853-3a14cc446409 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:47.441381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96822f1e-6d07-460f-b853-3a14cc446409 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:47.441684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96822f1e-6d07-460f-b853-3a14cc446409 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:47.443149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96822f1e-6d07-460f-b853-3a14cc446409 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:47.456887 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 629/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:47.462887 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e620f1-602a-4d1b-ba3b-a716feb8d28b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:47.552011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:47.578797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e620f1-602a-4d1b-ba3b-a716feb8d28b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:47.623368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515ff38-2406-471c-86ec-12497bd2e22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:47.624043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515ff38-2406-471c-86ec-12497bd2e22b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:47.624443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515ff38-2406-471c-86ec-12497bd2e22b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:47.624739 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515ff38-2406-471c-86ec-12497bd2e22b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:47.633562 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515ff38-2406-471c-86ec-12497bd2e22b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:47.635181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515ff38-2406-471c-86ec-12497bd2e22b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-9515ff38-2406-471c-86ec-12497bd2e22b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:21:47.648402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:47.649711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:47.650898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:21:47.650898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:21:47.650898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:21:47.650898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:21:47.650898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:21:47.650898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:21:47.650898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:21:47.650898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:21:47.650898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 615/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:21:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:47.656875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96822f1e-6d07-460f-b853-3a14cc446409 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:47.771587 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96822f1e-6d07-460f-b853-3a14cc446409 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:47.810381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec30ad-0253-4bcb-88bd-ded0655fbb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:47.811273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec30ad-0253-4bcb-88bd-ded0655fbb5c None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:47.811548 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec30ad-0253-4bcb-88bd-ded0655fbb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:47.811821 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec30ad-0253-4bcb-88bd-ded0655fbb5c None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:47.812558 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec30ad-0253-4bcb-88bd-ded0655fbb5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:47.812820 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec30ad-0253-4bcb-88bd-ded0655fbb5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:47.814344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec30ad-0253-4bcb-88bd-ded0655fbb5c None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:47.827196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 630/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:21:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:47.833395 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515ff38-2406-471c-86ec-12497bd2e22b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:47.947728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515ff38-2406-471c-86ec-12497bd2e22b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:47.984328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6fa19a-76d4-4566-b2da-64360c02627a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:47.985057 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fa19a-76d4-4566-b2da-64360c02627a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:47.985407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fa19a-76d4-4566-b2da-64360c02627a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:47.985732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fa19a-76d4-4566-b2da-64360c02627a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:47.986783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6fa19a-76d4-4566-b2da-64360c02627a None admin] RBAC: Adding query filter params (role.id=2b68b618a1b846f69bbe9aa08852a66f, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=784b6147de91435292e4cf3940bcad9c) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:47.987142 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6fa19a-76d4-4566-b2da-64360c02627a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:47.988848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6fa19a-76d4-4566-b2da-64360c02627a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:47.998857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 616/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:21:47 2022] GET /identity/v3/role_assignments?user.id=784b6147de91435292e4cf3940bcad9c&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=2b68b618a1b846f69bbe9aa08852a66f => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:50.291491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553a60b-f633-4abe-a367-631354b40f72 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:50.292031 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553a60b-f633-4abe-a367-631354b40f72 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:50.292031 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553a60b-f633-4abe-a367-631354b40f72 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:50.293524 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 631/1247] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:21:50.298821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca677414-623d-4895-9429-34648c0e6676 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:50.299407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca677414-623d-4895-9429-34648c0e6676 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:50.299526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca677414-623d-4895-9429-34648c0e6676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:50.306237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:50.320944 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ca677414-623d-4895-9429-34648c0e6676 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:21:50.322119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ca677414-623d-4895-9429-34648c0e6676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:21:50.401020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 617/1248] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:21:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:21:50.680487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553a60b-f633-4abe-a367-631354b40f72 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:50.701381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:50.798244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553a60b-f633-4abe-a367-631354b40f72 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:50.835319 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ee9eb-5c97-47b2-aaf0-383994af88ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:50.836014 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ee9eb-5c97-47b2-aaf0-383994af88ef None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:50.836316 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ee9eb-5c97-47b2-aaf0-383994af88ef None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:50.836589 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ee9eb-5c97-47b2-aaf0-383994af88ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:50.850294 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ee9eb-5c97-47b2-aaf0-383994af88ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-8d4ee9eb-5c97-47b2-aaf0-383994af88ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:21:50.855256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:50.856404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:50.857273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:50.857273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:50.857273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:50.857273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:50.857273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:21:50.857273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:21:50.857273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:21:50.857273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:21:50.857273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:21:50.857273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:21:50.857273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:21:50.857273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:21:50.857273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 632/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:21:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:50.863122 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca677414-623d-4895-9429-34648c0e6676 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:50.976266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca677414-623d-4895-9429-34648c0e6676 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:51.008436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:51.013845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b087c6-1b01-4c67-9249-5bfe901afaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:51.014472 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b087c6-1b01-4c67-9249-5bfe901afaa0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:51.014767 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b087c6-1b01-4c67-9249-5bfe901afaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:51.015038 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b087c6-1b01-4c67-9249-5bfe901afaa0 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:51.015710 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b087c6-1b01-4c67-9249-5bfe901afaa0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:51.016010 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b087c6-1b01-4c67-9249-5bfe901afaa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:51.017408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b087c6-1b01-4c67-9249-5bfe901afaa0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:51.025631 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 618/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:21:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:51.031741 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ee9eb-5c97-47b2-aaf0-383994af88ef None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:51.117066 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:51.143554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ee9eb-5c97-47b2-aaf0-383994af88ef None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:51.187540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22852ecf-a89a-4500-95b4-a70e20769ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:51.188303 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22852ecf-a89a-4500-95b4-a70e20769ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:51.188574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22852ecf-a89a-4500-95b4-a70e20769ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:51.188882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22852ecf-a89a-4500-95b4-a70e20769ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:51.200373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22852ecf-a89a-4500-95b4-a70e20769ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:51.201830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22852ecf-a89a-4500-95b4-a70e20769ec6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-22852ecf-a89a-4500-95b4-a70e20769ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:21:51.219358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:51.220680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:21:51.221701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:21:51.221701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:51.221701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:21:51.221701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 633/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:51.226895 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b087c6-1b01-4c67-9249-5bfe901afaa0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:51.348168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b087c6-1b01-4c67-9249-5bfe901afaa0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:51.388614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd9222-dd80-43ac-963c-773e2467489f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:51.389226 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd9222-dd80-43ac-963c-773e2467489f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:51.389500 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd9222-dd80-43ac-963c-773e2467489f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:51.389770 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd9222-dd80-43ac-963c-773e2467489f None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:51.390399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd9222-dd80-43ac-963c-773e2467489f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:51.390731 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd9222-dd80-43ac-963c-773e2467489f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:51.392101 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd9222-dd80-43ac-963c-773e2467489f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:51.406538 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 619/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:51.410871 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22852ecf-a89a-4500-95b4-a70e20769ec6 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:51.525200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22852ecf-a89a-4500-95b4-a70e20769ec6 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:51.552203 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:51.560380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f6168-015f-41ef-808c-5775d094dee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:51.561308 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f6168-015f-41ef-808c-5775d094dee9 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:51.561605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f6168-015f-41ef-808c-5775d094dee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:51.561881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f6168-015f-41ef-808c-5775d094dee9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:51.570675 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f6168-015f-41ef-808c-5775d094dee9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:51.572205 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f6168-015f-41ef-808c-5775d094dee9 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-4b4f6168-015f-41ef-808c-5775d094dee9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:21:51.583867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:51.584984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:51.586171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:51.586171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:51.586171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:21:51.586171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:21:51.586171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:21:51.586171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:21:51.586171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:21:51.586171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:21:51.586171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 16:21:51.586171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:21:51.586171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 634/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 16:21:51.593542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd9222-dd80-43ac-963c-773e2467489f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:51.659544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:51.708341 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd9222-dd80-43ac-963c-773e2467489f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:51.745817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bda86a-ca4a-4a21-aac7-a5c974e5d6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:51.746438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bda86a-ca4a-4a21-aac7-a5c974e5d6bb None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:51.746745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bda86a-ca4a-4a21-aac7-a5c974e5d6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:51.747017 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bda86a-ca4a-4a21-aac7-a5c974e5d6bb None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:51.747746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bda86a-ca4a-4a21-aac7-a5c974e5d6bb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:51.748050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bda86a-ca4a-4a21-aac7-a5c974e5d6bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:51.749605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bda86a-ca4a-4a21-aac7-a5c974e5d6bb None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:51.767000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 620/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Sep 9 16:21:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:51.773403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f6168-015f-41ef-808c-5775d094dee9 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:51.880262 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f6168-015f-41ef-808c-5775d094dee9 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:51.924601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b3946-0e8d-43cf-b009-a4fd83fbe18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:51.925283 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b3946-0e8d-43cf-b009-a4fd83fbe18f None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:51.925623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b3946-0e8d-43cf-b009-a4fd83fbe18f None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:51.925989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b3946-0e8d-43cf-b009-a4fd83fbe18f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:51.987281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:52.000660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b3946-0e8d-43cf-b009-a4fd83fbe18f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:52.002223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b3946-0e8d-43cf-b009-a4fd83fbe18f None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:52.104465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 635/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:21:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:21:52.110750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bda86a-ca4a-4a21-aac7-a5c974e5d6bb None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:52.218231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bda86a-ca4a-4a21-aac7-a5c974e5d6bb None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:52.231736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:52.256436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae726b4-71ed-4814-8559-af56ed6f454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:52.257186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae726b4-71ed-4814-8559-af56ed6f454f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:52.257362 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae726b4-71ed-4814-8559-af56ed6f454f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:52.257705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae726b4-71ed-4814-8559-af56ed6f454f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:52.268454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae726b4-71ed-4814-8559-af56ed6f454f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:52.269514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae726b4-71ed-4814-8559-af56ed6f454f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-0ae726b4-71ed-4814-8559-af56ed6f454f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:21:52.284679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:52.285906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:21:52.286827 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:21:52.286827 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:52.286827 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:21:52.286827 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 621/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:52.292092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b3946-0e8d-43cf-b009-a4fd83fbe18f None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:52.383500 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:52.396132 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b3946-0e8d-43cf-b009-a4fd83fbe18f None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:52.436033 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc9a1d-29e9-42ba-9a3a-dc660f788f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:52.436830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc9a1d-29e9-42ba-9a3a-dc660f788f28 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:52.437059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc9a1d-29e9-42ba-9a3a-dc660f788f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:52.437193 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc9a1d-29e9-42ba-9a3a-dc660f788f28 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:52.437970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc9a1d-29e9-42ba-9a3a-dc660f788f28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:52.438230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc9a1d-29e9-42ba-9a3a-dc660f788f28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:52.439864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc9a1d-29e9-42ba-9a3a-dc660f788f28 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:52.454465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 636/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:52.461157 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae726b4-71ed-4814-8559-af56ed6f454f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:52.567287 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae726b4-71ed-4814-8559-af56ed6f454f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:52.606242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19664d71-71d1-4f27-ad9f-1d1174de022f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:52.606968 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19664d71-71d1-4f27-ad9f-1d1174de022f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:52.607246 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19664d71-71d1-4f27-ad9f-1d1174de022f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:52.607514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19664d71-71d1-4f27-ad9f-1d1174de022f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:52.615184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19664d71-71d1-4f27-ad9f-1d1174de022f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:52.616667 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19664d71-71d1-4f27-ad9f-1d1174de022f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-19664d71-71d1-4f27-ad9f-1d1174de022f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:21:52.629180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:52.630279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:52.631228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:21:52.631228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:21:52.631228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:21:52.631228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:21:52.631228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:21:52.631228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:21:52.631228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:21:52.631228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:21:52.631228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 622/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:21:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:52.636702 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc9a1d-29e9-42ba-9a3a-dc660f788f28 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:52.749330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc9a1d-29e9-42ba-9a3a-dc660f788f28 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:52.786967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2aa10-d395-4260-969f-cce0db2a66aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:52.787586 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2aa10-d395-4260-969f-cce0db2a66aa None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:52.787861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2aa10-d395-4260-969f-cce0db2a66aa None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:52.788124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2aa10-d395-4260-969f-cce0db2a66aa None admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:52.788816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2aa10-d395-4260-969f-cce0db2a66aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:52.789114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2aa10-d395-4260-969f-cce0db2a66aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:52.790543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2aa10-d395-4260-969f-cce0db2a66aa None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:52.804209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 637/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:21:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:21:52.811208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19664d71-71d1-4f27-ad9f-1d1174de022f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:52.839922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:52.923759 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19664d71-71d1-4f27-ad9f-1d1174de022f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:52.963134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63da9f-43b4-464a-b660-9ffe16e35541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:52.963744 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63da9f-43b4-464a-b660-9ffe16e35541 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:52.964011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63da9f-43b4-464a-b660-9ffe16e35541 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:52.964277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63da9f-43b4-464a-b660-9ffe16e35541 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:52.964988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63da9f-43b4-464a-b660-9ffe16e35541 None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=784b6147de91435292e4cf3940bcad9c) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:52.965289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63da9f-43b4-464a-b660-9ffe16e35541 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:52.966746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63da9f-43b4-464a-b660-9ffe16e35541 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:52.972416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 623/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:21:52 2022] GET /identity/v3/role_assignments?user.id=784b6147de91435292e4cf3940bcad9c&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=c6da7ba2be504943af29338ee8855f05 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:55.088460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e2218-7d04-4665-b4c9-42d3d02417b4 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:55.089237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e2218-7d04-4665-b4c9-42d3d02417b4 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:55.089399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e2218-7d04-4665-b4c9-42d3d02417b4 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:55.091591 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 638/1261] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:21:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:21:55.099038 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd0bbd-1c18-413f-a3c3-796ca8fd8e33 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:55.099431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd0bbd-1c18-413f-a3c3-796ca8fd8e33 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:55.099587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd0bbd-1c18-413f-a3c3-796ca8fd8e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:55.119069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c3bd0bbd-1c18-413f-a3c3-796ca8fd8e33 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:21:55.120268 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c3bd0bbd-1c18-413f-a3c3-796ca8fd8e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:21:55.195755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 624/1262] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:21:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:21:55.472521 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e2218-7d04-4665-b4c9-42d3d02417b4 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:55.491607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:55.601228 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e2218-7d04-4665-b4c9-42d3d02417b4 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:55.643215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53910216-2da8-4def-9b91-69564a20ef85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:55.643841 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53910216-2da8-4def-9b91-69564a20ef85 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:55.644116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53910216-2da8-4def-9b91-69564a20ef85 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:55.644388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53910216-2da8-4def-9b91-69564a20ef85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:55.655191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53910216-2da8-4def-9b91-69564a20ef85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-53910216-2da8-4def-9b91-69564a20ef85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:21:55.659922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:55.660951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:55.662383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:55.662383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:55.662383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:55.662383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:55.662383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:21:55.662383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:21:55.662383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:21:55.662383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:21:55.662383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:21:55.662383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:21:55.662383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:21:55.662383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:21:55.662383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 639/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:21:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:55.670958 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd0bbd-1c18-413f-a3c3-796ca8fd8e33 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:55.691733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:55.780106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd0bbd-1c18-413f-a3c3-796ca8fd8e33 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:55.819829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d1aa1-c079-46dd-9d33-58347edff671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:55.820416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d1aa1-c079-46dd-9d33-58347edff671 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:55.820693 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d1aa1-c079-46dd-9d33-58347edff671 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:55.820962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d1aa1-c079-46dd-9d33-58347edff671 None admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:55.821617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d1aa1-c079-46dd-9d33-58347edff671 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:55.821905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d1aa1-c079-46dd-9d33-58347edff671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:55.823302 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d1aa1-c079-46dd-9d33-58347edff671 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:55.831340 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 625/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:21:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:55.838037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53910216-2da8-4def-9b91-69564a20ef85 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:55.910746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:55.950090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53910216-2da8-4def-9b91-69564a20ef85 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:55.986939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f57da-7fee-4eaa-bb39-254b17a8d073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:55.987562 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f57da-7fee-4eaa-bb39-254b17a8d073 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:55.987832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f57da-7fee-4eaa-bb39-254b17a8d073 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:55.988095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f57da-7fee-4eaa-bb39-254b17a8d073 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:55.998398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f57da-7fee-4eaa-bb39-254b17a8d073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:55.999867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f57da-7fee-4eaa-bb39-254b17a8d073 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-8b2f57da-7fee-4eaa-bb39-254b17a8d073 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:21:56.015377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:56.016443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:21:56.017507 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:21:56.017507 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:56.017507 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:21:56.018680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 640/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:56.022464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d1aa1-c079-46dd-9d33-58347edff671 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:56.108389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:56.129157 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d1aa1-c079-46dd-9d33-58347edff671 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:56.163930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2576e-bf0b-46ab-8344-ebf757638689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:56.164524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2576e-bf0b-46ab-8344-ebf757638689 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:56.164796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2576e-bf0b-46ab-8344-ebf757638689 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:56.165057 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2576e-bf0b-46ab-8344-ebf757638689 None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:56.165688 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2576e-bf0b-46ab-8344-ebf757638689 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:56.165982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2576e-bf0b-46ab-8344-ebf757638689 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:56.167387 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2576e-bf0b-46ab-8344-ebf757638689 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:56.179248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 626/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:56.186275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f57da-7fee-4eaa-bb39-254b17a8d073 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:56.294021 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f57da-7fee-4eaa-bb39-254b17a8d073 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:56.314276 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:56.328444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c545dc6-6901-47f3-9683-aaf841d90cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:56.329054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c545dc6-6901-47f3-9683-aaf841d90cdc None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:56.329327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c545dc6-6901-47f3-9683-aaf841d90cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:56.329597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c545dc6-6901-47f3-9683-aaf841d90cdc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:56.337010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c545dc6-6901-47f3-9683-aaf841d90cdc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:56.338387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c545dc6-6901-47f3-9683-aaf841d90cdc None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-3c545dc6-6901-47f3-9683-aaf841d90cdc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:21:56.349222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:56.350273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:56.351223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:56.351223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:56.351223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:21:56.351223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:21:56.351223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:21:56.351223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:21:56.351223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:21:56.351223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:21:56.351223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 16:21:56.351223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:21:56.351223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 641/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:56.356223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2576e-bf0b-46ab-8344-ebf757638689 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:56.465946 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2576e-bf0b-46ab-8344-ebf757638689 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:56.500415 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408c606-5f19-48e8-bcc5-b5cdb8fa513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:56.501009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408c606-5f19-48e8-bcc5-b5cdb8fa513f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:56.501277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408c606-5f19-48e8-bcc5-b5cdb8fa513f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:56.501544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408c606-5f19-48e8-bcc5-b5cdb8fa513f None admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:56.502297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408c606-5f19-48e8-bcc5-b5cdb8fa513f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:56.502617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408c606-5f19-48e8-bcc5-b5cdb8fa513f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:56.504048 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408c606-5f19-48e8-bcc5-b5cdb8fa513f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:56.512668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:56.521629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 627/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Sep 9 16:21:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:56.527928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c545dc6-6901-47f3-9683-aaf841d90cdc None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:56.628981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c545dc6-6901-47f3-9683-aaf841d90cdc None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:56.663196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478e21d-4ff3-4e1a-9d83-421489f6c3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:56.663840 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478e21d-4ff3-4e1a-9d83-421489f6c3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:56.664111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478e21d-4ff3-4e1a-9d83-421489f6c3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:56.664376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478e21d-4ff3-4e1a-9d83-421489f6c3c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:56.675378 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478e21d-4ff3-4e1a-9d83-421489f6c3c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:56.676802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478e21d-4ff3-4e1a-9d83-421489f6c3c0 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-c478e21d-4ff3-4e1a-9d83-421489f6c3c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:21:56.691798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:56.692873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:21:56.693887 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:21:56.693887 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:21:56.693887 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:21:56.693887 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 642/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:56.698757 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408c606-5f19-48e8-bcc5-b5cdb8fa513f None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:56.800103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408c606-5f19-48e8-bcc5-b5cdb8fa513f None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:56.839222 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6a635-7e98-4198-b467-55c58f69639a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:56.840157 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6a635-7e98-4198-b467-55c58f69639a None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:56.840575 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6a635-7e98-4198-b467-55c58f69639a None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:56.840988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6a635-7e98-4198-b467-55c58f69639a None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:56.842022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6a635-7e98-4198-b467-55c58f69639a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:56.842483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6a635-7e98-4198-b467-55c58f69639a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:56.844658 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6a635-7e98-4198-b467-55c58f69639a None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:56.856360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 628/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:21:56.862094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478e21d-4ff3-4e1a-9d83-421489f6c3c0 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:56.894862 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:56.962876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478e21d-4ff3-4e1a-9d83-421489f6c3c0 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:57.000060 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa120da-f901-4f96-a95a-a28564b87828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:57.000680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa120da-f901-4f96-a95a-a28564b87828 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:57.000950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa120da-f901-4f96-a95a-a28564b87828 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:57.001216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa120da-f901-4f96-a95a-a28564b87828 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:57.008172 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa120da-f901-4f96-a95a-a28564b87828 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:57.009607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa120da-f901-4f96-a95a-a28564b87828 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-ffa120da-f901-4f96-a95a-a28564b87828 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:21:57.021108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:21:57.022119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:21:57.023055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:21:57.023055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:21:57.023055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:21:57.023055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:21:57.023055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:21:57.023055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:21:57.023055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:21:57.023055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:21:57.023055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 643/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:21:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:21:57.028409 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6a635-7e98-4198-b467-55c58f69639a None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:57.093629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:57.137063 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6a635-7e98-4198-b467-55c58f69639a None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:57.177084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02814d74-d4b1-42db-8bf5-ed8f93cab626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:57.177625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02814d74-d4b1-42db-8bf5-ed8f93cab626 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:57.177899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02814d74-d4b1-42db-8bf5-ed8f93cab626 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:57.178162 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02814d74-d4b1-42db-8bf5-ed8f93cab626 None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:57.178880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02814d74-d4b1-42db-8bf5-ed8f93cab626 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:21:57.179181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02814d74-d4b1-42db-8bf5-ed8f93cab626 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:57.180597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02814d74-d4b1-42db-8bf5-ed8f93cab626 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:57.193138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 629/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:21:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:21:57.199376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa120da-f901-4f96-a95a-a28564b87828 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:57.298993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:57.308559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa120da-f901-4f96-a95a-a28564b87828 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:57.342457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0762ca-a3a5-4aad-a796-78e201652324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:57.343141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0762ca-a3a5-4aad-a796-78e201652324 None admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:57.343414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0762ca-a3a5-4aad-a796-78e201652324 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:57.343678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0762ca-a3a5-4aad-a796-78e201652324 None admin] PATH_INFO: `/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/784b6147de91435292e4cf3940bcad9c/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:57.345676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0762ca-a3a5-4aad-a796-78e201652324 None admin] RBAC: Authorizing `identity:create_grant(project_id=253ff30a56444df28d44c4b7eaadb276, user_id=784b6147de91435292e4cf3940bcad9c, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:21:57.347232 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0762ca-a3a5-4aad-a796-78e201652324 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:21:57.363003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 644/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Sep 9 16:21:57 2022] PUT /identity/v3/projects/253ff30a56444df28d44c4b7eaadb276/users/784b6147de91435292e4cf3940bcad9c/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:21:59.482401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855edc85-f648-4f1d-afc4-5b216f0b37c6 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:59.483002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855edc85-f648-4f1d-afc4-5b216f0b37c6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:59.483002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855edc85-f648-4f1d-afc4-5b216f0b37c6 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:59.484314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 630/1274] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:21:59.489643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa53e32-b65b-4bd1-8de7-430653d54d65 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:59.490113 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa53e32-b65b-4bd1-8de7-430653d54d65 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:59.490209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa53e32-b65b-4bd1-8de7-430653d54d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:21:59.509641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-cfa53e32-b65b-4bd1-8de7-430653d54d65 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:21:59.510880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cfa53e32-b65b-4bd1-8de7-430653d54d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:21:59.584164 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 645/1275] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:21:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:21:59.848464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855edc85-f648-4f1d-afc4-5b216f0b37c6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:21:59.962224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855edc85-f648-4f1d-afc4-5b216f0b37c6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:21:59.980534 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:21:59.997567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658320ef-f91e-4a5e-bdd9-573484e7dd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:21:59.998183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658320ef-f91e-4a5e-bdd9-573484e7dd9c None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:21:59.998455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658320ef-f91e-4a5e-bdd9-573484e7dd9c None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:21:59.998744 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658320ef-f91e-4a5e-bdd9-573484e7dd9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:00.009651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658320ef-f91e-4a5e-bdd9-573484e7dd9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-658320ef-f91e-4a5e-bdd9-573484e7dd9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 16:22:00.014354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:22:00.015440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:22:00.016420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:22:00.016420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:22:00.016420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:22:00.016420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:22:00.016420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 16:22:00.016420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 16:22:00.016420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 16:22:00.016420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 16:22:00.016420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 16:22:00.016420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 16:22:00.016420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 16:22:00.016420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:22:00.016420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 631/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Sep 9 16:21:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:22:00.022237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa53e32-b65b-4bd1-8de7-430653d54d65 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:00.118057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:00.139526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa53e32-b65b-4bd1-8de7-430653d54d65 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:00.178438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8266c-f223-4e5a-9962-93f303c87515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:00.179081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8266c-f223-4e5a-9962-93f303c87515 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:00.179359 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8266c-f223-4e5a-9962-93f303c87515 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:00.179629 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8266c-f223-4e5a-9962-93f303c87515 None admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:00.180295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8266c-f223-4e5a-9962-93f303c87515 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:22:00.180595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8266c-f223-4e5a-9962-93f303c87515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:00.181967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8266c-f223-4e5a-9962-93f303c87515 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:00.190208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 646/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Sep 9 16:22:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:22:00.198933 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658320ef-f91e-4a5e-bdd9-573484e7dd9c None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:00.305139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658320ef-f91e-4a5e-bdd9-573484e7dd9c None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:00.340979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff509e82-3be0-4814-969e-3a325f25e69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:00.341589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff509e82-3be0-4814-969e-3a325f25e69b None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:00.341869 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff509e82-3be0-4814-969e-3a325f25e69b None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:00.342138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff509e82-3be0-4814-969e-3a325f25e69b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:00.352778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff509e82-3be0-4814-969e-3a325f25e69b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:00.354221 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff509e82-3be0-4814-969e-3a325f25e69b None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-ff509e82-3be0-4814-969e-3a325f25e69b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:22:00.368926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:22:00.369918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:22:00.370930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:22:00.370930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:22:00.370930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:22:00.370930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 632/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:22:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:22:00.376009 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8266c-f223-4e5a-9962-93f303c87515 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:00.519111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8266c-f223-4e5a-9962-93f303c87515 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:00.553376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163da6f6-3c3c-480f-9f38-51435aad94d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:00.553977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163da6f6-3c3c-480f-9f38-51435aad94d6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:00.554259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163da6f6-3c3c-480f-9f38-51435aad94d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:00.554543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163da6f6-3c3c-480f-9f38-51435aad94d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:00.555261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163da6f6-3c3c-480f-9f38-51435aad94d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:22:00.555561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163da6f6-3c3c-480f-9f38-51435aad94d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:00.556944 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163da6f6-3c3c-480f-9f38-51435aad94d6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:00.569016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 647/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:22:00.578427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff509e82-3be0-4814-969e-3a325f25e69b None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:00.615826 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:00.688722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff509e82-3be0-4814-969e-3a325f25e69b None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:00.724913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0756b-3d0a-4c04-bd69-cb5c733cb4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:00.725538 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0756b-3d0a-4c04-bd69-cb5c733cb4ac None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:00.725814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0756b-3d0a-4c04-bd69-cb5c733cb4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:00.726079 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0756b-3d0a-4c04-bd69-cb5c733cb4ac None admin] PATH_INFO: `/v3/users/placement` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:00.733951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0756b-3d0a-4c04-bd69-cb5c733cb4ac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:00.735429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0756b-3d0a-4c04-bd69-cb5c733cb4ac None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-90e0756b-3d0a-4c04-bd69-cb5c733cb4ac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 16:22:00.746872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:22:00.748012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:22:00.749026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:22:00.749026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:22:00.749026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 16:22:00.749026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 16:22:00.749026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 16:22:00.749026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 16:22:00.749026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 16:22:00.749026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 16:22:00.749026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 16:22:00.749026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:22:00.749026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 633/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:22:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:22:00.754207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163da6f6-3c3c-480f-9f38-51435aad94d6 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:00.758411 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:00.866244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163da6f6-3c3c-480f-9f38-51435aad94d6 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:00.900705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a11927-3c1b-4a54-b6d7-83cd63c5e963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:00.901296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a11927-3c1b-4a54-b6d7-83cd63c5e963 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:00.901565 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a11927-3c1b-4a54-b6d7-83cd63c5e963 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:00.901833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a11927-3c1b-4a54-b6d7-83cd63c5e963 None admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:00.902541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a11927-3c1b-4a54-b6d7-83cd63c5e963 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:22:00.902864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a11927-3c1b-4a54-b6d7-83cd63c5e963 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:00.904278 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a11927-3c1b-4a54-b6d7-83cd63c5e963 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:00.921011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 648/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Sep 9 16:22:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:22:00.927332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0756b-3d0a-4c04-bd69-cb5c733cb4ac None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:01.021084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:01.030926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0756b-3d0a-4c04-bd69-cb5c733cb4ac None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:01.065223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea23807-4f4d-4bb4-afd5-bddbeb7ed714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:01.065811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea23807-4f4d-4bb4-afd5-bddbeb7ed714 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:01.066082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea23807-4f4d-4bb4-afd5-bddbeb7ed714 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:01.066348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea23807-4f4d-4bb4-afd5-bddbeb7ed714 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:01.143523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea23807-4f4d-4bb4-afd5-bddbeb7ed714 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:01.144988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea23807-4f4d-4bb4-afd5-bddbeb7ed714 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:01.241550 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:01.253741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 634/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Sep 9 16:22:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 16:22:01.259801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a11927-3c1b-4a54-b6d7-83cd63c5e963 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:01.323250 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:01.362812 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a11927-3c1b-4a54-b6d7-83cd63c5e963 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:01.397892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95771697-56b3-4deb-aa7a-8a8e1f03f7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:01.398542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95771697-56b3-4deb-aa7a-8a8e1f03f7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:01.398816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95771697-56b3-4deb-aa7a-8a8e1f03f7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:01.399084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95771697-56b3-4deb-aa7a-8a8e1f03f7d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:01.408909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95771697-56b3-4deb-aa7a-8a8e1f03f7d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:01.410430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95771697-56b3-4deb-aa7a-8a8e1f03f7d3 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-95771697-56b3-4deb-aa7a-8a8e1f03f7d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 16:22:01.425067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:22:01.426165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 16:22:01.427220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 16:22:01.427220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 16:22:01.427220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:22:01.430712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 649/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:22:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:22:01.431899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea23807-4f4d-4bb4-afd5-bddbeb7ed714 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:01.535031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea23807-4f4d-4bb4-afd5-bddbeb7ed714 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:01.571283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e68b3-a35a-4cd6-b2da-68348c2c63bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:01.571876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e68b3-a35a-4cd6-b2da-68348c2c63bf None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:01.572153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e68b3-a35a-4cd6-b2da-68348c2c63bf None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:01.572417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e68b3-a35a-4cd6-b2da-68348c2c63bf None admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:01.573049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e68b3-a35a-4cd6-b2da-68348c2c63bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:22:01.573415 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e68b3-a35a-4cd6-b2da-68348c2c63bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:01.574762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e68b3-a35a-4cd6-b2da-68348c2c63bf None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:01.585977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 635/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Sep 9 16:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:22:01.591860 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95771697-56b3-4deb-aa7a-8a8e1f03f7d3 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:01.697468 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95771697-56b3-4deb-aa7a-8a8e1f03f7d3 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:01.714756 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:01.737290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fd905-fccf-4815-930e-a6f87204ea26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:01.737948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fd905-fccf-4815-930e-a6f87204ea26 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:01.738210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fd905-fccf-4815-930e-a6f87204ea26 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:01.738478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fd905-fccf-4815-930e-a6f87204ea26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:01.745081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fd905-fccf-4815-930e-a6f87204ea26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:01.746517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fd905-fccf-4815-930e-a6f87204ea26 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application [None req-971fd905-fccf-4815-930e-a6f87204ea26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 16:22:01.757746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application with Lock( Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:22:01.758775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:22:01.759916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 16:22:01.759916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 16:22:01.759916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 16:22:01.759916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 16:22:01.759916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 16:22:01.759916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 16:22:01.759916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 16:22:01.759916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: ERROR keystone.server.flask.application Sep 09 16:22:01.759916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 650/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Sep 9 16:22:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:22:01.765127 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e68b3-a35a-4cd6-b2da-68348c2c63bf None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:01.825905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:01.877733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e68b3-a35a-4cd6-b2da-68348c2c63bf None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:01.912534 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ab6e9-fe7e-40cf-91f7-9d8dff638a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:01.913138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ab6e9-fe7e-40cf-91f7-9d8dff638a3f None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:01.913410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ab6e9-fe7e-40cf-91f7-9d8dff638a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:01.913674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ab6e9-fe7e-40cf-91f7-9d8dff638a3f None admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:01.914358 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ab6e9-fe7e-40cf-91f7-9d8dff638a3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:22:01.914687 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ab6e9-fe7e-40cf-91f7-9d8dff638a3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:01.916109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ab6e9-fe7e-40cf-91f7-9d8dff638a3f None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:01.927967 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 636/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Sep 9 16:22:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:22:01.934130 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fd905-fccf-4815-930e-a6f87204ea26 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:02.037340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fd905-fccf-4815-930e-a6f87204ea26 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:02.074166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3d243-f8c6-4b56-bf49-ef76d0a29c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:02.074807 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3d243-f8c6-4b56-bf49-ef76d0a29c26 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:02.075090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3d243-f8c6-4b56-bf49-ef76d0a29c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:02.075355 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3d243-f8c6-4b56-bf49-ef76d0a29c26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:02.076130 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3d243-f8c6-4b56-bf49-ef76d0a29c26 None admin] RBAC: Adding query filter params (role.id=c6da7ba2be504943af29338ee8855f05, scope.project.id=253ff30a56444df28d44c4b7eaadb276, user.id=784b6147de91435292e4cf3940bcad9c) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:22:02.076374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3d243-f8c6-4b56-bf49-ef76d0a29c26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:02.077803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3d243-f8c6-4b56-bf49-ef76d0a29c26 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:02.083732 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 651/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Sep 9 16:22:01 2022] GET /identity/v3/role_assignments?user.id=784b6147de91435292e4cf3940bcad9c&scope.project.id=253ff30a56444df28d44c4b7eaadb276&role.id=c6da7ba2be504943af29338ee8855f05 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:22:04.207860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e666016-41d0-4965-9876-b00726130870 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:04.208757 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e666016-41d0-4965-9876-b00726130870 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:04.208757 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e666016-41d0-4965-9876-b00726130870 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:04.210928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 637/1288] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:22:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:22:04.218915 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073692f-fd56-4aeb-949a-522405ac4ecd None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:04.219375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073692f-fd56-4aeb-949a-522405ac4ecd None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:04.219478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073692f-fd56-4aeb-949a-522405ac4ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:04.239516 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-3073692f-fd56-4aeb-949a-522405ac4ecd None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:22:04.240677 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-3073692f-fd56-4aeb-949a-522405ac4ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:22:04.274020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:04.314164 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 652/1289] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:22:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 16:22:04.582208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e666016-41d0-4965-9876-b00726130870 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:04.689264 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e666016-41d0-4965-9876-b00726130870 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:04.699850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:04.728766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73d9dd-485a-490e-bc3a-c35a4b5114f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:04.729391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d9dd-485a-490e-bc3a-c35a4b5114f0 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:04.729671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d9dd-485a-490e-bc3a-c35a4b5114f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:04.729938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d9dd-485a-490e-bc3a-c35a4b5114f0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:04.730540 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73d9dd-485a-490e-bc3a-c35a4b5114f0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:04.731922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73d9dd-485a-490e-bc3a-c35a4b5114f0 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-dd73d9dd-485a-490e-bc3a-c35a4b5114f0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:22:04.741441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:22:04.742718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:22:04.743585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 638/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Sep 9 16:22:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:22:04.748711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073692f-fd56-4aeb-949a-522405ac4ecd None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:04.852988 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073692f-fd56-4aeb-949a-522405ac4ecd None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:04.887554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7353062-6358-4919-84bf-7a610ce1b5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:04.888148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7353062-6358-4919-84bf-7a610ce1b5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:04.888420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7353062-6358-4919-84bf-7a610ce1b5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:04.888684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7353062-6358-4919-84bf-7a610ce1b5b8 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:04.889310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7353062-6358-4919-84bf-7a610ce1b5b8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:22:04.889613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7353062-6358-4919-84bf-7a610ce1b5b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:04.891008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7353062-6358-4919-84bf-7a610ce1b5b8 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:04.895851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 653/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Fri Sep 9 16:22:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:22:04.901735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73d9dd-485a-490e-bc3a-c35a4b5114f0 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:05.004408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73d9dd-485a-490e-bc3a-c35a4b5114f0 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:05.037899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc7e46-4dcb-46a4-82c2-b80d41f74221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:05.038477 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc7e46-4dcb-46a4-82c2-b80d41f74221 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:05.038779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc7e46-4dcb-46a4-82c2-b80d41f74221 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:05.039046 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc7e46-4dcb-46a4-82c2-b80d41f74221 None admin] PATH_INFO: `/v3/services` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:05.039655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc7e46-4dcb-46a4-82c2-b80d41f74221 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:22:05.039950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc7e46-4dcb-46a4-82c2-b80d41f74221 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:05.041295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc7e46-4dcb-46a4-82c2-b80d41f74221 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:05.045740 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 639/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Fri Sep 9 16:22:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:22:07.163529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fa6d5-ac7c-4080-ad51-fdf6cfc2bd7c None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:07.164320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fa6d5-ac7c-4080-ad51-fdf6cfc2bd7c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:07.164320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fa6d5-ac7c-4080-ad51-fdf6cfc2bd7c None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:07.166462 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 654/1293] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:22:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:22:07.171858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98b5a5-f51f-4ba8-b995-93a8b16112ae None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:07.172300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98b5a5-f51f-4ba8-b995-93a8b16112ae None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:07.172391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98b5a5-f51f-4ba8-b995-93a8b16112ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:07.197028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3c98b5a5-f51f-4ba8-b995-93a8b16112ae None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:22:07.198234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3c98b5a5-f51f-4ba8-b995-93a8b16112ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:22:07.233004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:07.278076 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 640/1294] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:22:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:22:07.545267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fa6d5-ac7c-4080-ad51-fdf6cfc2bd7c None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:07.585428 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:07.656502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fa6d5-ac7c-4080-ad51-fdf6cfc2bd7c None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:07.705923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4e4f1-d078-4ded-881e-e31eaf494cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:07.706542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4e4f1-d078-4ded-881e-e31eaf494cbd None admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:07.706838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4e4f1-d078-4ded-881e-e31eaf494cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:07.707109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4e4f1-d078-4ded-881e-e31eaf494cbd None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:07.707769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4e4f1-d078-4ded-881e-e31eaf494cbd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:07.709148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4e4f1-d078-4ded-881e-e31eaf494cbd None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:07.725842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 655/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Sep 9 16:22:07 2022] POST /identity/v3/services => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:22:09.875007 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aefc7d-949a-48fc-8e21-1b57aa1085e8 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:09.875573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aefc7d-949a-48fc-8e21-1b57aa1085e8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:09.875573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aefc7d-949a-48fc-8e21-1b57aa1085e8 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:09.876913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 641/1296] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:22:09.883385 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2697b7f-390c-445a-b388-a30841b9b381 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:09.884107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2697b7f-390c-445a-b388-a30841b9b381 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:09.884261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2697b7f-390c-445a-b388-a30841b9b381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:09.908264 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-e2697b7f-390c-445a-b388-a30841b9b381 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:22:09.909442 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e2697b7f-390c-445a-b388-a30841b9b381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:22:09.984847 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 656/1297] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:22:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:22:10.256275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aefc7d-949a-48fc-8e21-1b57aa1085e8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:10.370766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aefc7d-949a-48fc-8e21-1b57aa1085e8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:10.405539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10658ca-1cdb-468e-bbe9-2fab4868afc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:10.406180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10658ca-1cdb-468e-bbe9-2fab4868afc6 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:10.406453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10658ca-1cdb-468e-bbe9-2fab4868afc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:10.406762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10658ca-1cdb-468e-bbe9-2fab4868afc6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:10.407339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10658ca-1cdb-468e-bbe9-2fab4868afc6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:10.408743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10658ca-1cdb-468e-bbe9-2fab4868afc6 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-c10658ca-1cdb-468e-bbe9-2fab4868afc6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:22:10.417760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:22:10.418951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:22:10.420012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 642/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Sep 9 16:22:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:22:10.426162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2697b7f-390c-445a-b388-a30841b9b381 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:10.446950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:10.532668 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2697b7f-390c-445a-b388-a30841b9b381 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:10.578158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce035fdf-3bb3-4cbd-a185-e92f7f7916b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:10.578791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce035fdf-3bb3-4cbd-a185-e92f7f7916b6 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:10.579068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce035fdf-3bb3-4cbd-a185-e92f7f7916b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:10.579332 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce035fdf-3bb3-4cbd-a185-e92f7f7916b6 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:10.579964 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce035fdf-3bb3-4cbd-a185-e92f7f7916b6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:22:10.580254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce035fdf-3bb3-4cbd-a185-e92f7f7916b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:10.581742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce035fdf-3bb3-4cbd-a185-e92f7f7916b6 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:10.586892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 657/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Fri Sep 9 16:22:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:22:10.593286 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10658ca-1cdb-468e-bbe9-2fab4868afc6 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:10.627954 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:10.700103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10658ca-1cdb-468e-bbe9-2fab4868afc6 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:10.735852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af0179-a612-486f-8439-a6dba5c31195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:10.736443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af0179-a612-486f-8439-a6dba5c31195 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:10.736719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af0179-a612-486f-8439-a6dba5c31195 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:10.736992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af0179-a612-486f-8439-a6dba5c31195 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:10.737682 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af0179-a612-486f-8439-a6dba5c31195 None admin] RBAC: Adding query filter params (interface=public, service_id=66cf76b7521144aa92c5927248213514, region_id=RegionOne) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:22:10.738011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af0179-a612-486f-8439-a6dba5c31195 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:10.739419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af0179-a612-486f-8439-a6dba5c31195 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:10.744494 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 643/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Sep 9 16:22:10 2022] GET /identity/v3/endpoints?service_id=66cf76b7521144aa92c5927248213514&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:22:10.750996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce035fdf-3bb3-4cbd-a185-e92f7f7916b6 None admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:10.837233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:10.861009 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce035fdf-3bb3-4cbd-a185-e92f7f7916b6 None admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:10.901312 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3576a4-ed3c-46d2-8514-f0dce957f489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:10.901932 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3576a4-ed3c-46d2-8514-f0dce957f489 None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:10.902246 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3576a4-ed3c-46d2-8514-f0dce957f489 None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:10.902654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3576a4-ed3c-46d2-8514-f0dce957f489 None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:10.903571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3576a4-ed3c-46d2-8514-f0dce957f489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:10.905687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3576a4-ed3c-46d2-8514-f0dce957f489 None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:10.914337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 658/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Sep 9 16:22:10 2022] GET /identity/v3/services => generated 2191 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:22:13.016735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11837d84-41ff-42e1-b7fd-8a0c06fb7d6a None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:13.017635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11837d84-41ff-42e1-b7fd-8a0c06fb7d6a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:13.017635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11837d84-41ff-42e1-b7fd-8a0c06fb7d6a None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:13.019885 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 644/1302] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:22:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:22:13.026198 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a16ad5-79ee-4450-a400-adc2c163d5eb None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:13.026945 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a16ad5-79ee-4450-a400-adc2c163d5eb None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:13.027133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a16ad5-79ee-4450-a400-adc2c163d5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:13.049220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f4a16ad5-79ee-4450-a400-adc2c163d5eb None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:22:13.050411 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f4a16ad5-79ee-4450-a400-adc2c163d5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:22:13.124484 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 659/1303] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:22:13 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 16:22:13.388818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11837d84-41ff-42e1-b7fd-8a0c06fb7d6a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:13.494452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:13.502692 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11837d84-41ff-42e1-b7fd-8a0c06fb7d6a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:13.541597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4b35f-aff1-4270-8227-eb317d195060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:13.542225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4b35f-aff1-4270-8227-eb317d195060 None admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:13.542531 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4b35f-aff1-4270-8227-eb317d195060 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:13.542805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4b35f-aff1-4270-8227-eb317d195060 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:13.543385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4b35f-aff1-4270-8227-eb317d195060 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:13.544752 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4b35f-aff1-4270-8227-eb317d195060 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application [None req-9de4b35f-aff1-4270-8227-eb317d195060 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 16:22:13.558816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 16:22:13.560272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application with Lock( Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self._enter() Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application return self.creator() Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application created_value = creator( Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: ERROR keystone.server.flask.application Sep 09 16:22:13.561242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 645/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Sep 9 16:22:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 16:22:13.566597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a16ad5-79ee-4450-a400-adc2c163d5eb None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:13.650995 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:13.674971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a16ad5-79ee-4450-a400-adc2c163d5eb None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:13.709758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd034dac-4baa-42f2-befd-b386abfe1faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:13.710358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd034dac-4baa-42f2-befd-b386abfe1faf None admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:13.710678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd034dac-4baa-42f2-befd-b386abfe1faf None admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:13.710940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd034dac-4baa-42f2-befd-b386abfe1faf None admin] PATH_INFO: `/v3/services` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:13.711571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd034dac-4baa-42f2-befd-b386abfe1faf None admin] RBAC: Adding query filter params (name=placement) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:22:13.711866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd034dac-4baa-42f2-befd-b386abfe1faf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:13.713250 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd034dac-4baa-42f2-befd-b386abfe1faf None admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:13.718382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 660/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Fri Sep 9 16:22:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:22:13.724476 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4b35f-aff1-4270-8227-eb317d195060 None admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:22:13.823797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4b35f-aff1-4270-8227-eb317d195060 None admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:22:13.859413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab4401-3f9e-468b-b79b-23059a6a02c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:22:13.859998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab4401-3f9e-468b-b79b-23059a6a02c8 None admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:22:13.860278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab4401-3f9e-468b-b79b-23059a6a02c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:22:13.860546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab4401-3f9e-468b-b79b-23059a6a02c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:22:13.861194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab4401-3f9e-468b-b79b-23059a6a02c8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:22:13.862563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab4401-3f9e-468b-b79b-23059a6a02c8 None admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:22:13.899277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 646/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:22:13 2022] POST /identity/v3/endpoints => generated 331 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:22:23.715334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=30 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:22:23.893804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:17.508520 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5470d-193e-467d-8351-660e12d29e5a None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:17.508982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5470d-193e-467d-8351-660e12d29e5a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:17.509149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5470d-193e-467d-8351-660e12d29e5a None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:17.510383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 661/1307] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:23:17.518319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe16c4d-3dfc-4f64-9e2b-1f8a74bb97e2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:17.519930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe16c4d-3dfc-4f64-9e2b-1f8a74bb97e2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:17.520791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe16c4d-3dfc-4f64-9e2b-1f8a74bb97e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:17.645325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ffe16c4d-3dfc-4f64-9e2b-1f8a74bb97e2 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:23:17.646707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ffe16c4d-3dfc-4f64-9e2b-1f8a74bb97e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:17.762071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 647/1308] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:23:18.054774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd4beb-5037-4b0e-a9b4-2a558d10445c None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:18.055101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd4beb-5037-4b0e-a9b4-2a558d10445c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:18.055492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd4beb-5037-4b0e-a9b4-2a558d10445c None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:18.057338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 662/1309] 10.222.0.23 () {54 vars in 965 bytes} [Fri Sep 9 16:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:23:18.064140 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d484e6f-ec41-4fe3-82d7-5c8126eb281e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:18.064475 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d484e6f-ec41-4fe3-82d7-5c8126eb281e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:18.064749 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d484e6f-ec41-4fe3-82d7-5c8126eb281e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:18.149372 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-2d484e6f-ec41-4fe3-82d7-5c8126eb281e None None] MFA Rules not processed for user `aa65b6854dc54e289cc3e8e0b3746dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:23:18.150741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2d484e6f-ec41-4fe3-82d7-5c8126eb281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:18.173058 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:18.233413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 648/1310] 10.222.0.23 () {58 vars in 1062 bytes} [Fri Sep 9 16:23:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:23:18.243185 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd4beb-5037-4b0e-a9b4-2a558d10445c None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:23:18.244934 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-86dd4beb-5037-4b0e-a9b4-2a558d10445c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:18.360291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd4beb-5037-4b0e-a9b4-2a558d10445c None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:23:18.442953 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f2432-67ad-4f99-bf92-fde02cf488a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:23:18.443679 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f2432-67ad-4f99-bf92-fde02cf488a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:18.443965 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f2432-67ad-4f99-bf92-fde02cf488a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:18.444245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f2432-67ad-4f99-bf92-fde02cf488a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:18.445939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-234f2432-67ad-4f99-bf92-fde02cf488a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:18.504067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:18.561945 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f2432-67ad-4f99-bf92-fde02cf488a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:23:18.563451 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f2432-67ad-4f99-bf92-fde02cf488a0 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:23:18.588380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 663/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:23:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:23:21.045250 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698888bd-5665-4004-b9d6-3e93bf732586 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:21.046087 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698888bd-5665-4004-b9d6-3e93bf732586 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:21.046087 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698888bd-5665-4004-b9d6-3e93bf732586 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:21.048266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 649/1312] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:23:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:23:21.054904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d43a8b-9009-47f7-8a05-f8d253fd2947 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:21.055337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d43a8b-9009-47f7-8a05-f8d253fd2947 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:21.055552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d43a8b-9009-47f7-8a05-f8d253fd2947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:21.141169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-65d43a8b-9009-47f7-8a05-f8d253fd2947 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:23:21.142378 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-65d43a8b-9009-47f7-8a05-f8d253fd2947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:21.256858 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 664/1313] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:23:21.540604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698888bd-5665-4004-b9d6-3e93bf732586 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:23:21.543104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-698888bd-5665-4004-b9d6-3e93bf732586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:21.668905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698888bd-5665-4004-b9d6-3e93bf732586 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:23:21.743113 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:21.745471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a356d3f-5d90-4fae-93d7-9571dd668fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:23:21.746238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a356d3f-5d90-4fae-93d7-9571dd668fac service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:21.746535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a356d3f-5d90-4fae-93d7-9571dd668fac service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:21.746814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a356d3f-5d90-4fae-93d7-9571dd668fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:21.749324 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5a356d3f-5d90-4fae-93d7-9571dd668fac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:21.871542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a356d3f-5d90-4fae-93d7-9571dd668fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:23:21.873128 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a356d3f-5d90-4fae-93d7-9571dd668fac service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:23:21.893794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 650/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:23:24.355471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec909fdd-91f9-4ec0-9bb1-20433cc0cfa5 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:24.356327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec909fdd-91f9-4ec0-9bb1-20433cc0cfa5 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:24.356327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec909fdd-91f9-4ec0-9bb1-20433cc0cfa5 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:24.358890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 665/1315] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:23:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:23:24.366880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4b4eb-22f5-4c4c-a90c-306cd6b71f51 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:24.367319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4b4eb-22f5-4c4c-a90c-306cd6b71f51 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:24.367410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4b4eb-22f5-4c4c-a90c-306cd6b71f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:24.434052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7de4b4eb-22f5-4c4c-a90c-306cd6b71f51 None None] MFA Rules not processed for user `6ca6948ab5ab429ab41c690eaa6cc6c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:23:24.435295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7de4b4eb-22f5-4c4c-a90c-306cd6b71f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:24.489073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:24.546604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 651/1316] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:23:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:23:24.829021 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec909fdd-91f9-4ec0-9bb1-20433cc0cfa5 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:23:24.831545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ec909fdd-91f9-4ec0-9bb1-20433cc0cfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:24.842327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:24.938991 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec909fdd-91f9-4ec0-9bb1-20433cc0cfa5 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:23:25.014878 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300355e6-903a-49c4-9fff-c065ad859c58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:23:25.015485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300355e6-903a-49c4-9fff-c065ad859c58 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:25.015760 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300355e6-903a-49c4-9fff-c065ad859c58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:25.016032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300355e6-903a-49c4-9fff-c065ad859c58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:25.017630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-300355e6-903a-49c4-9fff-c065ad859c58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:25.083016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:25.133574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300355e6-903a-49c4-9fff-c065ad859c58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:23:25.135050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300355e6-903a-49c4-9fff-c065ad859c58 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:23:25.152901 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 666/1317] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:23:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:23:27.797435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9710c5-1978-4840-87d5-30cc30acf206 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:27.798350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9710c5-1978-4840-87d5-30cc30acf206 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:27.798350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9710c5-1978-4840-87d5-30cc30acf206 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:27.800485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 652/1318] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:23:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:23:27.806093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4d074-8ad0-4e4f-b273-dfe651cf7171 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:27.806560 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4d074-8ad0-4e4f-b273-dfe651cf7171 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:27.806669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4d074-8ad0-4e4f-b273-dfe651cf7171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:27.877183 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-97d4d074-8ad0-4e4f-b273-dfe651cf7171 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:23:27.878380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-97d4d074-8ad0-4e4f-b273-dfe651cf7171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:27.977977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 667/1319] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:23:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:23:28.274084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9710c5-1978-4840-87d5-30cc30acf206 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:23:28.276547 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bf9710c5-1978-4840-87d5-30cc30acf206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:28.394050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9710c5-1978-4840-87d5-30cc30acf206 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:23:28.470355 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a78ebb-6744-4255-a458-26917404bcfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:23:28.471021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a78ebb-6744-4255-a458-26917404bcfd service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:28.471298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a78ebb-6744-4255-a458-26917404bcfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:28.471568 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a78ebb-6744-4255-a458-26917404bcfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:28.473229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-40a78ebb-6744-4255-a458-26917404bcfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:28.481180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:28.594786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a78ebb-6744-4255-a458-26917404bcfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:23:28.596223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a78ebb-6744-4255-a458-26917404bcfd service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:23:28.614788 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 653/1320] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:23:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:23:31.395054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4554e84-c160-40c7-bdb8-6707fa787f9d None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:31.395653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4554e84-c160-40c7-bdb8-6707fa787f9d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:31.395653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4554e84-c160-40c7-bdb8-6707fa787f9d None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:31.397067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 668/1321] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:23:31.402614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a206e1-c4d7-4204-b909-ee35b4b16409 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:31.403018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a206e1-c4d7-4204-b909-ee35b4b16409 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:31.403211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a206e1-c4d7-4204-b909-ee35b4b16409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:31.477450 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-52a206e1-c4d7-4204-b909-ee35b4b16409 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:23:31.478745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-52a206e1-c4d7-4204-b909-ee35b4b16409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:31.542134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:31.586208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 654/1322] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:23:31.884749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841a4b5-2ac5-41f0-80ab-f87e14185ecb None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:31.884980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841a4b5-2ac5-41f0-80ab-f87e14185ecb None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:31.885285 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841a4b5-2ac5-41f0-80ab-f87e14185ecb None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:31.886848 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 669/1323] 10.222.0.23 () {54 vars in 965 bytes} [Fri Sep 9 16:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:23:31.893243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f45689-ebb0-4820-89c4-2cace1b7862c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:31.893588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f45689-ebb0-4820-89c4-2cace1b7862c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:31.893927 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f45689-ebb0-4820-89c4-2cace1b7862c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:31.977387 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-50f45689-ebb0-4820-89c4-2cace1b7862c None None] MFA Rules not processed for user `aa65b6854dc54e289cc3e8e0b3746dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:23:31.978649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-50f45689-ebb0-4820-89c4-2cace1b7862c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:32.056280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 655/1324] 10.222.0.23 () {58 vars in 1062 bytes} [Fri Sep 9 16:23:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:23:32.065799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841a4b5-2ac5-41f0-80ab-f87e14185ecb None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:23:32.067376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8841a4b5-2ac5-41f0-80ab-f87e14185ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:32.089215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:32.175464 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841a4b5-2ac5-41f0-80ab-f87e14185ecb None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:23:32.250171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb9789-d056-4a9d-ab45-d174fe0da056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:23:32.250818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb9789-d056-4a9d-ab45-d174fe0da056 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:32.251097 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb9789-d056-4a9d-ab45-d174fe0da056 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:32.251366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb9789-d056-4a9d-ab45-d174fe0da056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:32.252962 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-6ebb9789-d056-4a9d-ab45-d174fe0da056 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:32.325308 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:32.364736 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb9789-d056-4a9d-ab45-d174fe0da056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:23:32.366173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb9789-d056-4a9d-ab45-d174fe0da056 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:23:32.384572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 670/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:23:34.767736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae769bb-6c63-4c9b-a809-1b32554d2404 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:34.768303 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae769bb-6c63-4c9b-a809-1b32554d2404 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:34.768303 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae769bb-6c63-4c9b-a809-1b32554d2404 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:34.769721 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 656/1326] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:23:34.775201 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa79db-a1b4-4926-bf16-4606fc87ce73 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:34.775608 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa79db-a1b4-4926-bf16-4606fc87ce73 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:34.775688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa79db-a1b4-4926-bf16-4606fc87ce73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:34.847578 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-83aa79db-a1b4-4926-bf16-4606fc87ce73 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:23:34.848898 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-83aa79db-a1b4-4926-bf16-4606fc87ce73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:34.946431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 671/1327] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:23:35.224055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae769bb-6c63-4c9b-a809-1b32554d2404 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:23:35.225655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-aae769bb-6c63-4c9b-a809-1b32554d2404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:35.290252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:35.335416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae769bb-6c63-4c9b-a809-1b32554d2404 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:23:35.409660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe29e9-2cf1-49ed-85b7-e198b556fd87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:23:35.410278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe29e9-2cf1-49ed-85b7-e198b556fd87 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:35.410542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe29e9-2cf1-49ed-85b7-e198b556fd87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:35.410832 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe29e9-2cf1-49ed-85b7-e198b556fd87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:35.412429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5dfe29e9-2cf1-49ed-85b7-e198b556fd87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:35.519354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:35.521580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe29e9-2cf1-49ed-85b7-e198b556fd87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:23:35.523107 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe29e9-2cf1-49ed-85b7-e198b556fd87 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:23:35.542052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 657/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:23:37.771660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9d382-53e9-404c-a2ca-0f1bffec216c None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:37.772216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9d382-53e9-404c-a2ca-0f1bffec216c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:37.772216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9d382-53e9-404c-a2ca-0f1bffec216c None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:37.773578 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 672/1329] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:23:37.779374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163449f-c490-4523-b855-85099106b397 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:37.779916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163449f-c490-4523-b855-85099106b397 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:37.779916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163449f-c490-4523-b855-85099106b397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:37.854049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a163449f-c490-4523-b855-85099106b397 None None] MFA Rules not processed for user `6ca6948ab5ab429ab41c690eaa6cc6c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:23:37.855286 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a163449f-c490-4523-b855-85099106b397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:37.954798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 658/1330] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:23:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:23:38.247674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9d382-53e9-404c-a2ca-0f1bffec216c None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:23:38.249211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e2c9d382-53e9-404c-a2ca-0f1bffec216c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:38.280061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:38.353029 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9d382-53e9-404c-a2ca-0f1bffec216c None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:23:38.436547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c034d-e8d6-48e0-8d75-046669554e06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:23:38.437173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c034d-e8d6-48e0-8d75-046669554e06 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:38.437451 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c034d-e8d6-48e0-8d75-046669554e06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:38.437714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c034d-e8d6-48e0-8d75-046669554e06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:38.439363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-372c034d-e8d6-48e0-8d75-046669554e06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:38.520631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:38.554505 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c034d-e8d6-48e0-8d75-046669554e06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:23:38.556003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c034d-e8d6-48e0-8d75-046669554e06 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:23:38.575277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 673/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:23:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:23:41.086390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb9171-685d-435c-aece-b4080e2b1562 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:41.086913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb9171-685d-435c-aece-b4080e2b1562 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:41.086913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb9171-685d-435c-aece-b4080e2b1562 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:41.088326 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 659/1332] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:23:41.094490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff78c9-c31c-4c77-9231-859c2e95eacf None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:41.095202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff78c9-c31c-4c77-9231-859c2e95eacf None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:41.095319 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff78c9-c31c-4c77-9231-859c2e95eacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:41.165910 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-90ff78c9-c31c-4c77-9231-859c2e95eacf None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:23:41.167193 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-90ff78c9-c31c-4c77-9231-859c2e95eacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:41.270344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 674/1333] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:23:41.551555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb9171-685d-435c-aece-b4080e2b1562 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:23:41.553222 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-77cb9171-685d-435c-aece-b4080e2b1562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:41.633353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:41.668285 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb9171-685d-435c-aece-b4080e2b1562 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:23:41.743625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35d371-923a-438c-bb3e-860536e7fd4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:23:41.744212 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35d371-923a-438c-bb3e-860536e7fd4f service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:41.744490 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35d371-923a-438c-bb3e-860536e7fd4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:41.744748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35d371-923a-438c-bb3e-860536e7fd4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:41.746313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4c35d371-923a-438c-bb3e-860536e7fd4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:41.870313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35d371-923a-438c-bb3e-860536e7fd4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:23:41.871799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35d371-923a-438c-bb3e-860536e7fd4f service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:23:41.882761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:41.894264 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 660/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:23:44.058891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9a39d-052b-4fc1-9237-784ea090b7e9 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:44.059499 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9a39d-052b-4fc1-9237-784ea090b7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:44.059499 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9a39d-052b-4fc1-9237-784ea090b7e9 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:44.060785 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 675/1335] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:23:44.067465 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea2196-4b17-4ee7-9ed9-432461cb4c4b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:44.067929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea2196-4b17-4ee7-9ed9-432461cb4c4b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:44.068035 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea2196-4b17-4ee7-9ed9-432461cb4c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:44.141594 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5bea2196-4b17-4ee7-9ed9-432461cb4c4b None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:23:44.142815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5bea2196-4b17-4ee7-9ed9-432461cb4c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:44.245816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 661/1336] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:23:44.527589 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9a39d-052b-4fc1-9237-784ea090b7e9 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:23:44.529189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-97a9a39d-052b-4fc1-9237-784ea090b7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:44.572151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:44.637767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9a39d-052b-4fc1-9237-784ea090b7e9 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:23:44.715152 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f0b03-33db-4a3d-bd58-43c26f527b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:23:44.715755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f0b03-33db-4a3d-bd58-43c26f527b73 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:44.716029 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f0b03-33db-4a3d-bd58-43c26f527b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:44.716294 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f0b03-33db-4a3d-bd58-43c26f527b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:44.717856 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a50f0b03-33db-4a3d-bd58-43c26f527b73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:44.805339 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:44.832613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f0b03-33db-4a3d-bd58-43c26f527b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:23:44.834052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f0b03-33db-4a3d-bd58-43c26f527b73 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:23:44.852671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 676/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:23:47.321385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f43c6-8c08-4344-9752-8f31e7307a4e None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:47.322270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f43c6-8c08-4344-9752-8f31e7307a4e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:47.322270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f43c6-8c08-4344-9752-8f31e7307a4e None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:47.324499 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 662/1338] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:23:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:23:47.332398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff17e2-4c9e-4815-94ba-7301819273ff None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:47.332841 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff17e2-4c9e-4815-94ba-7301819273ff None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:47.332931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff17e2-4c9e-4815-94ba-7301819273ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:47.400097 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-76ff17e2-4c9e-4815-94ba-7301819273ff None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:23:47.401287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-76ff17e2-4c9e-4815-94ba-7301819273ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:47.507131 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 677/1339] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:23:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:23:47.797607 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f43c6-8c08-4344-9752-8f31e7307a4e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:23:47.800058 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-039f43c6-8c08-4344-9752-8f31e7307a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:47.885635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:47.914369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f43c6-8c08-4344-9752-8f31e7307a4e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:23:47.990473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0a680-4333-498c-a978-c05b8d9a2456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:23:47.991104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0a680-4333-498c-a978-c05b8d9a2456 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:47.991376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0a680-4333-498c-a978-c05b8d9a2456 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:47.991662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0a680-4333-498c-a978-c05b8d9a2456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:47.993276 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-27d0a680-4333-498c-a978-c05b8d9a2456 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:48.112708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0a680-4333-498c-a978-c05b8d9a2456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:23:48.114271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0a680-4333-498c-a978-c05b8d9a2456 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:23:48.133669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 663/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:23:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:23:52.406537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7834e0-5903-4451-b8c4-18d02ba26f3c None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:52.407162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7834e0-5903-4451-b8c4-18d02ba26f3c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:52.407162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7834e0-5903-4451-b8c4-18d02ba26f3c None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:52.408433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 678/1341] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:23:52.414021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83693fd3-2bdb-491c-b69d-ce572ecea092 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:52.414487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83693fd3-2bdb-491c-b69d-ce572ecea092 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:52.414608 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83693fd3-2bdb-491c-b69d-ce572ecea092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:52.420416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:52.484922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-83693fd3-2bdb-491c-b69d-ce572ecea092 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:23:52.486046 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-83693fd3-2bdb-491c-b69d-ce572ecea092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:52.585639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 664/1342] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:23:52.861833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7834e0-5903-4451-b8c4-18d02ba26f3c None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:23:52.863585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7b7834e0-5903-4451-b8c4-18d02ba26f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:52.917373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:52.975317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7834e0-5903-4451-b8c4-18d02ba26f3c None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:23:53.053652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28fc0c-fd98-4558-b4f7-84e5c5062e0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:23:53.054216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28fc0c-fd98-4558-b4f7-84e5c5062e0c service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:53.054520 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28fc0c-fd98-4558-b4f7-84e5c5062e0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:53.054806 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28fc0c-fd98-4558-b4f7-84e5c5062e0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:53.056815 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-4f28fc0c-fd98-4558-b4f7-84e5c5062e0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:53.156358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:53.173424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28fc0c-fd98-4558-b4f7-84e5c5062e0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:23:53.174941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28fc0c-fd98-4558-b4f7-84e5c5062e0c service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:23:53.194042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 679/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:23:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:23:55.974055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9d331-ca1a-4839-88e2-2b7986f40303 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:55.974684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9d331-ca1a-4839-88e2-2b7986f40303 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:55.974684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9d331-ca1a-4839-88e2-2b7986f40303 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:55.975975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 665/1344] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:23:55.981323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b014d0-c83a-415c-a3bb-87675ee9aac9 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:55.981766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b014d0-c83a-415c-a3bb-87675ee9aac9 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:55.981890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b014d0-c83a-415c-a3bb-87675ee9aac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:56.049761 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-38b014d0-c83a-415c-a3bb-87675ee9aac9 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:23:56.051134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-38b014d0-c83a-415c-a3bb-87675ee9aac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:56.152454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 680/1345] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:23:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:23:56.432125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9d331-ca1a-4839-88e2-2b7986f40303 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:23:56.434144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-90c9d331-ca1a-4839-88e2-2b7986f40303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:56.531569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:23:56.550336 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9d331-ca1a-4839-88e2-2b7986f40303 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:23:56.627850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32154193-6385-444c-a58b-39cad2c56fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:23:56.628374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32154193-6385-444c-a58b-39cad2c56fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:23:56.628646 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32154193-6385-444c-a58b-39cad2c56fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:23:56.628909 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32154193-6385-444c-a58b-39cad2c56fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:23:56.630646 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-32154193-6385-444c-a58b-39cad2c56fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:23:56.743134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32154193-6385-444c-a58b-39cad2c56fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:23:56.744581 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32154193-6385-444c-a58b-39cad2c56fb6 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:23:56.762848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 666/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:23:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:24:01.155309 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85258147-f9bf-4f5f-a51d-e02231ba2531 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:01.155819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85258147-f9bf-4f5f-a51d-e02231ba2531 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:01.155919 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85258147-f9bf-4f5f-a51d-e02231ba2531 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:01.157632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 681/1347] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:24:01.162842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691cb6c-3b90-47a9-a30d-220965098b59 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:01.163308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691cb6c-3b90-47a9-a30d-220965098b59 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:01.163429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691cb6c-3b90-47a9-a30d-220965098b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:01.193449 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:01.243691 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7691cb6c-3b90-47a9-a30d-220965098b59 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:01.245090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7691cb6c-3b90-47a9-a30d-220965098b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:01.342761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 667/1348] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:01.633994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85258147-f9bf-4f5f-a51d-e02231ba2531 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:01.635586 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-85258147-f9bf-4f5f-a51d-e02231ba2531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:01.695532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:01.749467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85258147-f9bf-4f5f-a51d-e02231ba2531 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:01.826950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab78224-546c-408f-9cba-01e31c4315b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:01.827528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab78224-546c-408f-9cba-01e31c4315b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:01.827835 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab78224-546c-408f-9cba-01e31c4315b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:01.828071 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab78224-546c-408f-9cba-01e31c4315b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:01.829678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cab78224-546c-408f-9cba-01e31c4315b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:01.936961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:01.940655 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab78224-546c-408f-9cba-01e31c4315b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:01.942405 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab78224-546c-408f-9cba-01e31c4315b1 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:01.960566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 682/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:24:06.921082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c0c34-7445-4153-b6de-adfa977adfd6 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:06.921590 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c0c34-7445-4153-b6de-adfa977adfd6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:06.921701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c0c34-7445-4153-b6de-adfa977adfd6 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:06.923326 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 668/1350] 10.222.0.23 () {54 vars in 921 bytes} [Fri Sep 9 16:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:06.930026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83b173-5a8f-401b-8569-63cf2955dc09 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:06.930431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83b173-5a8f-401b-8569-63cf2955dc09 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:06.931032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83b173-5a8f-401b-8569-63cf2955dc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:07.056868 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-be83b173-5a8f-401b-8569-63cf2955dc09 None None] MFA Rules not processed for user `784b6147de91435292e4cf3940bcad9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:07.058077 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-be83b173-5a8f-401b-8569-63cf2955dc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:07.172673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 683/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Sep 9 16:24:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:11.225521 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7040cf2-a60b-4702-9cac-a282388b0f9f None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:11.226096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7040cf2-a60b-4702-9cac-a282388b0f9f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:11.226096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7040cf2-a60b-4702-9cac-a282388b0f9f None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:11.227612 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 669/1352] 10.222.0.23 () {54 vars in 921 bytes} [Fri Sep 9 16:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:11.232411 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194ad98-b84d-4470-b9a6-f434ee40a37a None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:11.232884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194ad98-b84d-4470-b9a6-f434ee40a37a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:11.232980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194ad98-b84d-4470-b9a6-f434ee40a37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:11.291366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:11.314020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-0194ad98-b84d-4470-b9a6-f434ee40a37a None None] MFA Rules not processed for user `784b6147de91435292e4cf3940bcad9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:11.315275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0194ad98-b84d-4470-b9a6-f434ee40a37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:11.341480 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=59 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:11.424180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 684/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Sep 9 16:24:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:12.830558 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cafd24-bf90-4121-8ecc-0068d4e35e64 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:12.831104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cafd24-bf90-4121-8ecc-0068d4e35e64 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:12.831232 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cafd24-bf90-4121-8ecc-0068d4e35e64 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:12.832975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 670/1354] 10.222.0.23 () {54 vars in 921 bytes} [Fri Sep 9 16:24:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:12.838109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81cc48-1c6e-40f0-b95d-5078b67a4fd4 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:12.838544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81cc48-1c6e-40f0-b95d-5078b67a4fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:12.838767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81cc48-1c6e-40f0-b95d-5078b67a4fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:12.917901 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-7c81cc48-1c6e-40f0-b95d-5078b67a4fd4 None None] MFA Rules not processed for user `784b6147de91435292e4cf3940bcad9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:12.919155 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7c81cc48-1c6e-40f0-b95d-5078b67a4fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:12.960903 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:13.028721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 685/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Sep 9 16:24:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:24:16.503489 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587b9b-4c11-40d2-9b8b-76043f885bf8 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:16.503989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587b9b-4c11-40d2-9b8b-76043f885bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:16.503989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587b9b-4c11-40d2-9b8b-76043f885bf8 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:16.506252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 671/1356] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:16.517236 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15110f76-2010-4da4-92aa-6c471b2731b2 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:16.518299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15110f76-2010-4da4-92aa-6c471b2731b2 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:16.518833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15110f76-2010-4da4-92aa-6c471b2731b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:16.603237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-15110f76-2010-4da4-92aa-6c471b2731b2 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:16.604611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-15110f76-2010-4da4-92aa-6c471b2731b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:16.709039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 686/1357] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:17.005072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aeaa81-25d4-48e5-b108-3720961a6548 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:17.006143 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aeaa81-25d4-48e5-b108-3720961a6548 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:17.006805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aeaa81-25d4-48e5-b108-3720961a6548 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:17.008746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 672/1358] 10.222.0.23 () {54 vars in 962 bytes} [Fri Sep 9 16:24:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:17.018214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafff1b-ecfe-47bb-b2ef-b7f58f07f1b3 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:17.019293 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafff1b-ecfe-47bb-b2ef-b7f58f07f1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:17.020039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafff1b-ecfe-47bb-b2ef-b7f58f07f1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:17.045922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:17.114003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-2dafff1b-ecfe-47bb-b2ef-b7f58f07f1b3 None None] MFA Rules not processed for user `152a41ad598b4b01b1881d928c53c848`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:17.115227 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2dafff1b-ecfe-47bb-b2ef-b7f58f07f1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:17.233797 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 687/1359] 10.222.0.23 () {58 vars in 1059 bytes} [Fri Sep 9 16:24:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:17.241744 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aeaa81-25d4-48e5-b108-3720961a6548 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:17.243267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-75aeaa81-25d4-48e5-b108-3720961a6548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:17.395514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aeaa81-25d4-48e5-b108-3720961a6548 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:17.473258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:17.521227 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573d71a-879c-4f21-8a4f-e1ec168e3bd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:17.521905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573d71a-879c-4f21-8a4f-e1ec168e3bd5 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:17.522182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573d71a-879c-4f21-8a4f-e1ec168e3bd5 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:17.522355 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573d71a-879c-4f21-8a4f-e1ec168e3bd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:17.524016 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e573d71a-879c-4f21-8a4f-e1ec168e3bd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:17.636463 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573d71a-879c-4f21-8a4f-e1ec168e3bd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:17.638808 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573d71a-879c-4f21-8a4f-e1ec168e3bd5 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:17.670710 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 673/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:24:20.037039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2575c0d-e13e-4b62-9242-e2804669b3a4 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:20.037631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2575c0d-e13e-4b62-9242-e2804669b3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:20.037631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2575c0d-e13e-4b62-9242-e2804669b3a4 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:20.039135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 688/1361] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:20.044348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c488a-cf5a-4a60-ae1b-39f03afa7329 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:20.044799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c488a-cf5a-4a60-ae1b-39f03afa7329 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:20.044799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c488a-cf5a-4a60-ae1b-39f03afa7329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:20.112368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-632c488a-cf5a-4a60-ae1b-39f03afa7329 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:20.113323 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-632c488a-cf5a-4a60-ae1b-39f03afa7329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:20.151281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:20.216241 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 674/1362] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:20.501683 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2575c0d-e13e-4b62-9242-e2804669b3a4 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:20.503452 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d2575c0d-e13e-4b62-9242-e2804669b3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:20.573954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:20.672836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2575c0d-e13e-4b62-9242-e2804669b3a4 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:20.809948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f98bfc-0bb6-49b9-943a-6d1288b89b7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:20.810622 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f98bfc-0bb6-49b9-943a-6d1288b89b7d service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:20.810908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f98bfc-0bb6-49b9-943a-6d1288b89b7d service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:20.811583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f98bfc-0bb6-49b9-943a-6d1288b89b7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:20.812765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-45f98bfc-0bb6-49b9-943a-6d1288b89b7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:20.841586 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:20.912378 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f98bfc-0bb6-49b9-943a-6d1288b89b7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:20.913970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f98bfc-0bb6-49b9-943a-6d1288b89b7d service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:20.932624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 689/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:24:23.382805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71186436-ff6f-4189-9643-46dae72ca258 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:23.383435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71186436-ff6f-4189-9643-46dae72ca258 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:23.383435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71186436-ff6f-4189-9643-46dae72ca258 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:23.384665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 675/1364] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:23.390111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bbcab-f36d-426b-8494-423cbfe44a37 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:23.390927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bbcab-f36d-426b-8494-423cbfe44a37 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:23.391180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bbcab-f36d-426b-8494-423cbfe44a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:23.460281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f23bbcab-f36d-426b-8494-423cbfe44a37 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:23.461475 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f23bbcab-f36d-426b-8494-423cbfe44a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:23.560629 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:23.570796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 690/1365] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:24:23.855594 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e57c3e-d10a-49f0-8ac7-a3343ee29b30 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:23.856453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e57c3e-d10a-49f0-8ac7-a3343ee29b30 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:23.856822 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e57c3e-d10a-49f0-8ac7-a3343ee29b30 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:23.858698 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 676/1366] 10.222.0.23 () {54 vars in 962 bytes} [Fri Sep 9 16:24:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:23.864542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1e634-e630-4fd2-80e2-f7e6759f05b5 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:23.864995 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1e634-e630-4fd2-80e2-f7e6759f05b5 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:23.865283 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1e634-e630-4fd2-80e2-f7e6759f05b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:23.947454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-46c1e634-e630-4fd2-80e2-f7e6759f05b5 None None] MFA Rules not processed for user `152a41ad598b4b01b1881d928c53c848`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:23.948860 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-46c1e634-e630-4fd2-80e2-f7e6759f05b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:24.069725 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 691/1367] 10.222.0.23 () {58 vars in 1059 bytes} [Fri Sep 9 16:24:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:24.077911 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e57c3e-d10a-49f0-8ac7-a3343ee29b30 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:24.082406 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-97e57c3e-d10a-49f0-8ac7-a3343ee29b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:24.243257 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e57c3e-d10a-49f0-8ac7-a3343ee29b30 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:24.265124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:24.365270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a76fc2-67fb-49ed-84f9-d5ca4ae18c1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:24.366039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a76fc2-67fb-49ed-84f9-d5ca4ae18c1e service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:24.366415 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a76fc2-67fb-49ed-84f9-d5ca4ae18c1e service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:24.366814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a76fc2-67fb-49ed-84f9-d5ca4ae18c1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:24.368544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d6a76fc2-67fb-49ed-84f9-d5ca4ae18c1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:24.464564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a76fc2-67fb-49ed-84f9-d5ca4ae18c1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:24.466206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a76fc2-67fb-49ed-84f9-d5ca4ae18c1e service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:24.486587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 677/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:24:24.618797 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a9d10-1128-419c-a564-eddf496cbcfe None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:24.619352 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a9d10-1128-419c-a564-eddf496cbcfe None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:24.619352 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a9d10-1128-419c-a564-eddf496cbcfe None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:24.620861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 692/1369] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:24.626867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e0ac3-9199-4952-b334-fbc5d87e701a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:24.627108 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e0ac3-9199-4952-b334-fbc5d87e701a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:24.627402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e0ac3-9199-4952-b334-fbc5d87e701a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:24.662938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:24.711537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-051e0ac3-9199-4952-b334-fbc5d87e701a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:24.713297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-051e0ac3-9199-4952-b334-fbc5d87e701a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:24.840928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 678/1370] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:25.180477 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b150b-81c7-401a-840b-776707f09d95 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:25.180477 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b150b-81c7-401a-840b-776707f09d95 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:25.180477 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b150b-81c7-401a-840b-776707f09d95 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:25.183495 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 693/1371] 10.222.0.23 () {54 vars in 964 bytes} [Fri Sep 9 16:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:25.186724 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f725b52-a9e0-43d6-bcc8-e8a9e6b59820 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:25.186963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f725b52-a9e0-43d6-bcc8-e8a9e6b59820 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:25.187258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f725b52-a9e0-43d6-bcc8-e8a9e6b59820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:25.275258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0f725b52-a9e0-43d6-bcc8-e8a9e6b59820 None None] MFA Rules not processed for user `95b665b9e27144b5bd684ece7ff465e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:25.276546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0f725b52-a9e0-43d6-bcc8-e8a9e6b59820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:25.327308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:25.366583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 679/1372] 10.222.0.23 () {58 vars in 1061 bytes} [Fri Sep 9 16:24:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:25.372552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b150b-81c7-401a-840b-776707f09d95 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:25.374104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-918b150b-81c7-401a-840b-776707f09d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:25.431415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:25.493500 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b150b-81c7-401a-840b-776707f09d95 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:25.576590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ffb4e-4d66-4e0d-b3db-af7b54eb505e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:25.577201 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ffb4e-4d66-4e0d-b3db-af7b54eb505e service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:25.577501 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ffb4e-4d66-4e0d-b3db-af7b54eb505e service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:25.577776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ffb4e-4d66-4e0d-b3db-af7b54eb505e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:25.579428 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e17ffb4e-4d66-4e0d-b3db-af7b54eb505e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:25.689611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:25.705763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ffb4e-4d66-4e0d-b3db-af7b54eb505e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:25.707292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ffb4e-4d66-4e0d-b3db-af7b54eb505e service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:25.729003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 694/1373] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:24:26.701145 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115006b-bfaa-4ac1-944c-954b6659c35c None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:26.701750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115006b-bfaa-4ac1-944c-954b6659c35c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:26.701750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115006b-bfaa-4ac1-944c-954b6659c35c None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:26.703078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 680/1374] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:26.708598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6bf7c-fc7e-4fec-aca4-93859959089c None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:26.709026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6bf7c-fc7e-4fec-aca4-93859959089c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:26.709144 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6bf7c-fc7e-4fec-aca4-93859959089c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:26.776450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-63b6bf7c-fc7e-4fec-aca4-93859959089c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:26.777720 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-63b6bf7c-fc7e-4fec-aca4-93859959089c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:26.883793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 695/1375] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:27.170347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115006b-bfaa-4ac1-944c-954b6659c35c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:27.172089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6115006b-bfaa-4ac1-944c-954b6659c35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:27.338863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115006b-bfaa-4ac1-944c-954b6659c35c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:27.392601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:27.457408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49b345-84be-4524-b9fb-29bb6ee14bd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:27.458018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49b345-84be-4524-b9fb-29bb6ee14bd5 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:27.458413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49b345-84be-4524-b9fb-29bb6ee14bd5 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:27.458722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49b345-84be-4524-b9fb-29bb6ee14bd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:27.460317 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7a49b345-84be-4524-b9fb-29bb6ee14bd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:27.548855 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49b345-84be-4524-b9fb-29bb6ee14bd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:27.550734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49b345-84be-4524-b9fb-29bb6ee14bd5 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:27.569157 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 681/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:24:29.755393 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6ffcc-8b22-4812-a454-bfa326da3bb3 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:29.755961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6ffcc-8b22-4812-a454-bfa326da3bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:29.755961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6ffcc-8b22-4812-a454-bfa326da3bb3 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:29.757622 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 696/1377] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:29.763183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8cb46-0070-4433-bda5-9c5a7eaca9a2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:29.763652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8cb46-0070-4433-bda5-9c5a7eaca9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:29.763741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8cb46-0070-4433-bda5-9c5a7eaca9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:29.831728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-30f8cb46-0070-4433-bda5-9c5a7eaca9a2 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:29.832923 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-30f8cb46-0070-4433-bda5-9c5a7eaca9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:29.840304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:29.938571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 682/1378] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:30.225983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6ffcc-8b22-4812-a454-bfa326da3bb3 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:30.227805 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-efe6ffcc-8b22-4812-a454-bfa326da3bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:30.292489 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:30.383225 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6ffcc-8b22-4812-a454-bfa326da3bb3 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:30.506776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd9d69-8048-44ce-b722-c25dfb9beb11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:30.507588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd9d69-8048-44ce-b722-c25dfb9beb11 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:30.507964 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd9d69-8048-44ce-b722-c25dfb9beb11 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:30.508335 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd9d69-8048-44ce-b722-c25dfb9beb11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:30.510028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d5bd9d69-8048-44ce-b722-c25dfb9beb11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:30.534942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:30.611839 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd9d69-8048-44ce-b722-c25dfb9beb11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:30.613597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd9d69-8048-44ce-b722-c25dfb9beb11 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:30.632047 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 697/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:24:32.806776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d172f-b978-4611-8650-079aea08fea5 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:32.807374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d172f-b978-4611-8650-079aea08fea5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:32.807374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d172f-b978-4611-8650-079aea08fea5 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:32.808855 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 683/1380] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:24:32.814114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3eabda-3320-438c-bc26-4f5093c06207 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:32.814578 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3eabda-3320-438c-bc26-4f5093c06207 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:32.814826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3eabda-3320-438c-bc26-4f5093c06207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:32.883826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-cf3eabda-3320-438c-bc26-4f5093c06207 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:32.885018 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cf3eabda-3320-438c-bc26-4f5093c06207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:32.979990 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:33.004575 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 698/1381] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:33.279018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d172f-b978-4611-8650-079aea08fea5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:33.280562 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9e6d172f-b978-4611-8650-079aea08fea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:33.426860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:33.439342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d172f-b978-4611-8650-079aea08fea5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:33.559659 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7de02-342a-4281-b603-4380079baa3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:33.560269 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7de02-342a-4281-b603-4380079baa3d service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:33.560540 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7de02-342a-4281-b603-4380079baa3d service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:33.560810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7de02-342a-4281-b603-4380079baa3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:33.562388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1ca7de02-342a-4281-b603-4380079baa3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:33.653884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7de02-342a-4281-b603-4380079baa3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:33.655768 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7de02-342a-4281-b603-4380079baa3d service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:33.668795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:33.680561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 684/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:24:36.016344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f301d0c-36a4-48c4-8f88-3e7a355a4bd5 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:36.016897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f301d0c-36a4-48c4-8f88-3e7a355a4bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:36.016897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f301d0c-36a4-48c4-8f88-3e7a355a4bd5 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:36.018267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 699/1383] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:36.023762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17807d78-1542-47b4-ba5c-176ac212bb98 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:36.024181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17807d78-1542-47b4-ba5c-176ac212bb98 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:36.024314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17807d78-1542-47b4-ba5c-176ac212bb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:36.104563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-17807d78-1542-47b4-ba5c-176ac212bb98 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:36.105796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-17807d78-1542-47b4-ba5c-176ac212bb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:36.205793 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 685/1384] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:36.493904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f301d0c-36a4-48c4-8f88-3e7a355a4bd5 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:36.495517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5f301d0c-36a4-48c4-8f88-3e7a355a4bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:36.657330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f301d0c-36a4-48c4-8f88-3e7a355a4bd5 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:36.726006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:36.771674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d6983-aa9b-4d84-af4a-5fd4abaff156 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:36.772219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d6983-aa9b-4d84-af4a-5fd4abaff156 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:36.772480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d6983-aa9b-4d84-af4a-5fd4abaff156 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:36.772773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d6983-aa9b-4d84-af4a-5fd4abaff156 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:36.774398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-138d6983-aa9b-4d84-af4a-5fd4abaff156 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:36.867611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d6983-aa9b-4d84-af4a-5fd4abaff156 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:36.869268 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d6983-aa9b-4d84-af4a-5fd4abaff156 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:36.887694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 700/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:24:39.048082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb2e4b-9da2-45a7-acd4-3d2a4f8da50e None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:39.048959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb2e4b-9da2-45a7-acd4-3d2a4f8da50e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:39.048959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb2e4b-9da2-45a7-acd4-3d2a4f8da50e None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:39.050159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 686/1386] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:24:39.056664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa72064-75d4-41a4-a608-33df6fff9cf7 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:39.057258 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa72064-75d4-41a4-a608-33df6fff9cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:39.057258 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa72064-75d4-41a4-a608-33df6fff9cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:39.129214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-baa72064-75d4-41a4-a608-33df6fff9cf7 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:39.130435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-baa72064-75d4-41a4-a608-33df6fff9cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:39.170695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:39.246606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 701/1387] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:39.533610 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb2e4b-9da2-45a7-acd4-3d2a4f8da50e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:39.535353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0fdb2e4b-9da2-45a7-acd4-3d2a4f8da50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:39.616732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:39.687803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb2e4b-9da2-45a7-acd4-3d2a4f8da50e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:39.803054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47bd70-6b75-4fc3-9119-ef2feb552134 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:39.803600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47bd70-6b75-4fc3-9119-ef2feb552134 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:39.803871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47bd70-6b75-4fc3-9119-ef2feb552134 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:39.804138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47bd70-6b75-4fc3-9119-ef2feb552134 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:39.806153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3a47bd70-6b75-4fc3-9119-ef2feb552134 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:39.850430 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:39.906203 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47bd70-6b75-4fc3-9119-ef2feb552134 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:39.907899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47bd70-6b75-4fc3-9119-ef2feb552134 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:39.926791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 687/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:24:42.126185 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36511a0-9e42-48bb-9298-4a3a882571f9 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:42.126737 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36511a0-9e42-48bb-9298-4a3a882571f9 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:42.126737 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36511a0-9e42-48bb-9298-4a3a882571f9 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:42.128490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 702/1389] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:42.133877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db351ea-de11-4aa2-b40e-cd41b1f1116e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:42.134368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db351ea-de11-4aa2-b40e-cd41b1f1116e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:42.134473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db351ea-de11-4aa2-b40e-cd41b1f1116e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:42.201578 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0db351ea-de11-4aa2-b40e-cd41b1f1116e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:42.202838 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0db351ea-de11-4aa2-b40e-cd41b1f1116e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:42.301998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 688/1390] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:42.591170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36511a0-9e42-48bb-9298-4a3a882571f9 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:42.592709 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e36511a0-9e42-48bb-9298-4a3a882571f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:42.749697 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36511a0-9e42-48bb-9298-4a3a882571f9 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:42.768302 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:42.869457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b899c5-2257-45eb-ab47-5d4d7a4fe205 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:42.870072 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b899c5-2257-45eb-ab47-5d4d7a4fe205 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:42.870347 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b899c5-2257-45eb-ab47-5d4d7a4fe205 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:42.870651 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b899c5-2257-45eb-ab47-5d4d7a4fe205 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:42.872247 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-14b899c5-2257-45eb-ab47-5d4d7a4fe205 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:42.966596 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b899c5-2257-45eb-ab47-5d4d7a4fe205 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:42.968269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b899c5-2257-45eb-ab47-5d4d7a4fe205 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:42.985939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 703/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:24:44.119250 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a46af7-108a-4752-8e59-8db80c970f8d None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:44.119845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a46af7-108a-4752-8e59-8db80c970f8d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:44.119845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a46af7-108a-4752-8e59-8db80c970f8d None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:44.121277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 689/1392] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:44.126614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee4120-a6a6-4b17-b67b-6b3a140704f1 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:44.127087 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee4120-a6a6-4b17-b67b-6b3a140704f1 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:44.127208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee4120-a6a6-4b17-b67b-6b3a140704f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:44.152302 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:44.195407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-e3ee4120-a6a6-4b17-b67b-6b3a140704f1 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:44.196600 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e3ee4120-a6a6-4b17-b67b-6b3a140704f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:44.294353 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 704/1393] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:44.591258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fd8ea-4d2e-44aa-9103-83cc6f440a87 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:44.591506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fd8ea-4d2e-44aa-9103-83cc6f440a87 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:44.591772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fd8ea-4d2e-44aa-9103-83cc6f440a87 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:44.593152 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 690/1394] 10.222.0.23 () {54 vars in 966 bytes} [Fri Sep 9 16:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:44.598761 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92522e7-e0d3-410c-9a14-8598793ffb56 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:44.599046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92522e7-e0d3-410c-9a14-8598793ffb56 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:44.599320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92522e7-e0d3-410c-9a14-8598793ffb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:44.680622 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f92522e7-e0d3-410c-9a14-8598793ffb56 None None] MFA Rules not processed for user `62a2fb70c9ae4f4b81ac4de513b868cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:44.681848 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f92522e7-e0d3-410c-9a14-8598793ffb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:44.722858 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:44.760059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 705/1395] 10.222.0.23 () {58 vars in 1063 bytes} [Fri Sep 9 16:24:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:44.769402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fd8ea-4d2e-44aa-9103-83cc6f440a87 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:44.771054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a39fd8ea-4d2e-44aa-9103-83cc6f440a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:44.790205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:44.883570 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fd8ea-4d2e-44aa-9103-83cc6f440a87 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:44.959003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac229b9d-b1a5-4ec8-b26e-251aa2c44725 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a2fb70c9ae4f4b81ac4de513b868cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:44.959722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac229b9d-b1a5-4ec8-b26e-251aa2c44725 service glance] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:44.960002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac229b9d-b1a5-4ec8-b26e-251aa2c44725 service glance] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:44.960269 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac229b9d-b1a5-4ec8-b26e-251aa2c44725 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:44.961852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ac229b9d-b1a5-4ec8-b26e-251aa2c44725 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:45.030384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:45.079711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac229b9d-b1a5-4ec8-b26e-251aa2c44725 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:45.081211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac229b9d-b1a5-4ec8-b26e-251aa2c44725 service glance] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:45.103064 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 691/1396] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:24:45.166850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce36bb7-9dfe-4a29-9934-f9c42fcfe235 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:45.167420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce36bb7-9dfe-4a29-9934-f9c42fcfe235 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:45.167516 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce36bb7-9dfe-4a29-9934-f9c42fcfe235 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:45.168999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 706/1397] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:45.174695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d521ebd-adcd-4dd8-bc59-8abf2f0c6842 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:45.175100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d521ebd-adcd-4dd8-bc59-8abf2f0c6842 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:45.175273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d521ebd-adcd-4dd8-bc59-8abf2f0c6842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:45.244606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3d521ebd-adcd-4dd8-bc59-8abf2f0c6842 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:45.245801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3d521ebd-adcd-4dd8-bc59-8abf2f0c6842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:45.354202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 692/1398] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:45.639428 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce36bb7-9dfe-4a29-9934-f9c42fcfe235 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:45.640963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2ce36bb7-9dfe-4a29-9934-f9c42fcfe235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:45.816984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce36bb7-9dfe-4a29-9934-f9c42fcfe235 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:45.858811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933ff6a-f71b-4fb3-89a8-c8728687cf45 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:45.859149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933ff6a-f71b-4fb3-89a8-c8728687cf45 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:45.859431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933ff6a-f71b-4fb3-89a8-c8728687cf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:45.872048 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:45.887763 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:45.929852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1933ff6a-f71b-4fb3-89a8-c8728687cf45 None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:45.931074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1933ff6a-f71b-4fb3-89a8-c8728687cf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:45.950082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2e0a8-118b-416f-99ba-b7a0aa4f98b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:45.950727 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2e0a8-118b-416f-99ba-b7a0aa4f98b8 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:45.951002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2e0a8-118b-416f-99ba-b7a0aa4f98b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:45.951265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2e0a8-118b-416f-99ba-b7a0aa4f98b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:45.952871 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8ff2e0a8-118b-416f-99ba-b7a0aa4f98b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:46.022744 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 693/1399] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:24:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:24:46.060001 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2e0a8-118b-416f-99ba-b7a0aa4f98b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:46.061669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2e0a8-118b-416f-99ba-b7a0aa4f98b8 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:46.068639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fd68b-d4c5-4b8f-9363-577cb27e4b1b None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:46.068921 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fd68b-d4c5-4b8f-9363-577cb27e4b1b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:46.069194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fd68b-d4c5-4b8f-9363-577cb27e4b1b None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:46.070736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 694/1400] 10.222.0.23 () {54 vars in 943 bytes} [Fri Sep 9 16:24:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:24:46.082418 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355e359-fdd2-4c3d-8512-767ed5e5cb58 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:46.082418 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355e359-fdd2-4c3d-8512-767ed5e5cb58 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:46.082418 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355e359-fdd2-4c3d-8512-767ed5e5cb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:46.086869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 707/1401] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:24:46.162710 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-2355e359-fdd2-4c3d-8512-767ed5e5cb58 None None] MFA Rules not processed for user `e926034945b44087b378b3468954a18a`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:46.163937 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2355e359-fdd2-4c3d-8512-767ed5e5cb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:46.226432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:46.290713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 695/1402] 10.222.0.23 () {58 vars in 1040 bytes} [Fri Sep 9 16:24:46 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:24:46.300489 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2e0a8-118b-416f-99ba-b7a0aa4f98b8 service nova] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:46.301734 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8ff2e0a8-118b-416f-99ba-b7a0aa4f98b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:46.362073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:46.395907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2e0a8-118b-416f-99ba-b7a0aa4f98b8 service nova] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:46.501284 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5dad3f-dce1-431f-b817-350d2b165859 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:46.501704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dad3f-dce1-431f-b817-350d2b165859 service swift] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:46.501986 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dad3f-dce1-431f-b817-350d2b165859 service swift] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:46.502254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dad3f-dce1-431f-b817-350d2b165859 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:46.504008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cf5dad3f-dce1-431f-b817-350d2b165859 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:46.592143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5dad3f-dce1-431f-b817-350d2b165859 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:46.593723 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5dad3f-dce1-431f-b817-350d2b165859 service swift] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:46.598856 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:46.605526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 708/1403] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:24:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:24:48.317841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198bb6-84d2-4aec-b068-dc143a2f4ea9 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:48.318405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198bb6-84d2-4aec-b068-dc143a2f4ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:48.318537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198bb6-84d2-4aec-b068-dc143a2f4ea9 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:48.319888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 696/1404] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:48.325537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a8835-91ba-4ee2-9ba9-1c08bbcc9ae3 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:48.326044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a8835-91ba-4ee2-9ba9-1c08bbcc9ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:48.326269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a8835-91ba-4ee2-9ba9-1c08bbcc9ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:48.414366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f52a8835-91ba-4ee2-9ba9-1c08bbcc9ae3 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:48.415733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f52a8835-91ba-4ee2-9ba9-1c08bbcc9ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:48.521578 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 709/1405] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:48.810386 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96198bb6-84d2-4aec-b068-dc143a2f4ea9 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:48.811950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-96198bb6-84d2-4aec-b068-dc143a2f4ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:48.977566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96198bb6-84d2-4aec-b068-dc143a2f4ea9 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:49.008948 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:49.122650 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e0b6d-addd-4c51-94c8-01efa52d9d78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:49.124597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e0b6d-addd-4c51-94c8-01efa52d9d78 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:49.124597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e0b6d-addd-4c51-94c8-01efa52d9d78 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:49.124597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e0b6d-addd-4c51-94c8-01efa52d9d78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:49.125627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-805e0b6d-addd-4c51-94c8-01efa52d9d78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:49.230763 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e0b6d-addd-4c51-94c8-01efa52d9d78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:49.232500 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e0b6d-addd-4c51-94c8-01efa52d9d78 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:49.252059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 697/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:24:51.461888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20478f4-8e52-4a1e-8b6d-92b91d2446cf None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:51.462538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20478f4-8e52-4a1e-8b6d-92b91d2446cf None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:51.462538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20478f4-8e52-4a1e-8b6d-92b91d2446cf None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:51.463797 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 710/1407] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:24:51.469234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e2458-3aac-4419-84fd-8090d067a82a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:51.469680 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e2458-3aac-4419-84fd-8090d067a82a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:51.469786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e2458-3aac-4419-84fd-8090d067a82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:51.513902 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:51.542691 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-306e2458-3aac-4419-84fd-8090d067a82a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:24:51.544010 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-306e2458-3aac-4419-84fd-8090d067a82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:51.644535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 698/1408] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:24:51.926687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20478f4-8e52-4a1e-8b6d-92b91d2446cf None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:24:51.928294 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c20478f4-8e52-4a1e-8b6d-92b91d2446cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:52.014736 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:52.089083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20478f4-8e52-4a1e-8b6d-92b91d2446cf None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:24:52.214171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10d1d3-5070-46b8-af9d-08dad593054a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:24:52.214805 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10d1d3-5070-46b8-af9d-08dad593054a service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:24:52.215084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10d1d3-5070-46b8-af9d-08dad593054a service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:24:52.216009 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10d1d3-5070-46b8-af9d-08dad593054a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:24:52.217715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5a10d1d3-5070-46b8-af9d-08dad593054a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:24:52.263450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:24:52.321084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10d1d3-5070-46b8-af9d-08dad593054a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:24:52.322844 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10d1d3-5070-46b8-af9d-08dad593054a service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:24:52.341756 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 711/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:25:01.643031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=51 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:02.339427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=27 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:09.125603 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e755095-bc0a-43dd-8c27-fa7c1af113f1 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:09.126162 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e755095-bc0a-43dd-8c27-fa7c1af113f1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:09.126162 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e755095-bc0a-43dd-8c27-fa7c1af113f1 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:09.127619 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 699/1410] 10.222.0.23 () {54 vars in 918 bytes} [Fri Sep 9 16:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:09.134112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c82e6a-04a5-4669-bcf3-dfa625783b25 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:09.134909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c82e6a-04a5-4669-bcf3-dfa625783b25 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:09.135051 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c82e6a-04a5-4669-bcf3-dfa625783b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:09.217362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-b6c82e6a-04a5-4669-bcf3-dfa625783b25 None None] MFA Rules not processed for user `784b6147de91435292e4cf3940bcad9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:09.218654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b6c82e6a-04a5-4669-bcf3-dfa625783b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:09.329399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 712/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Fri Sep 9 16:25:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:09.344246 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58146f74-baff-4d6b-8a97-1a5ca4541636 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:09.344527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58146f74-baff-4d6b-8a97-1a5ca4541636 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:09.344793 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58146f74-baff-4d6b-8a97-1a5ca4541636 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:09.346142 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 700/1412] 10.222.0.23 () {54 vars in 918 bytes} [Fri Sep 9 16:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:09.350898 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d6270-a671-42aa-bb2b-f9b06e7d9447 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:09.351184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d6270-a671-42aa-bb2b-f9b06e7d9447 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:09.351452 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d6270-a671-42aa-bb2b-f9b06e7d9447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:09.425636 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:09.429223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-597d6270-a671-42aa-bb2b-f9b06e7d9447 None None] MFA Rules not processed for user `152a41ad598b4b01b1881d928c53c848`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:09.431714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-597d6270-a671-42aa-bb2b-f9b06e7d9447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:09.552933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 713/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Fri Sep 9 16:25:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:25:12.132535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff3191-4a20-423b-9659-aa2783e30cd6 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:12.133391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff3191-4a20-423b-9659-aa2783e30cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:12.133391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff3191-4a20-423b-9659-aa2783e30cd6 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:12.135659 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:25:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:12.142215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924f520-cdf4-4231-a612-4506b23cfbf5 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:12.142994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924f520-cdf4-4231-a612-4506b23cfbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:12.143144 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924f520-cdf4-4231-a612-4506b23cfbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:12.211802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-3924f520-cdf4-4231-a612-4506b23cfbf5 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:12.213000 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-3924f520-cdf4-4231-a612-4506b23cfbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:12.275209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:12.320949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:12.617995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff3191-4a20-423b-9659-aa2783e30cd6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:12.620381 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-85ff3191-4a20-423b-9659-aa2783e30cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:12.778434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff3191-4a20-423b-9659-aa2783e30cd6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:12.858647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:13.026967 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50706b49-80dc-4ee5-8700-a76db82ad77e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:13.027566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50706b49-80dc-4ee5-8700-a76db82ad77e service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:13.027843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50706b49-80dc-4ee5-8700-a76db82ad77e service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:13.028082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50706b49-80dc-4ee5-8700-a76db82ad77e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:13.029689 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-50706b49-80dc-4ee5-8700-a76db82ad77e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:13.125342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50706b49-80dc-4ee5-8700-a76db82ad77e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:13.126902 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50706b49-80dc-4ee5-8700-a76db82ad77e service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:13.145753 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 702/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:14.279056 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e24fe8-7a98-44c3-b4f2-fd8b8c5707ea None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:14.279879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e24fe8-7a98-44c3-b4f2-fd8b8c5707ea None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:14.279879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e24fe8-7a98-44c3-b4f2-fd8b8c5707ea None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:14.281938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 16:25:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:14.288380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754173f-6e1e-4bb2-947c-7c9d4d7e52dc None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:14.288855 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754173f-6e1e-4bb2-947c-7c9d4d7e52dc None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:14.288966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754173f-6e1e-4bb2-947c-7c9d4d7e52dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:14.366489 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b754173f-6e1e-4bb2-947c-7c9d4d7e52dc None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:14.367713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b754173f-6e1e-4bb2-947c-7c9d4d7e52dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:14.395694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:14.474230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 16:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:14.633869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e24fe8-7a98-44c3-b4f2-fd8b8c5707ea None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:14.635390 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-03e24fe8-7a98-44c3-b4f2-fd8b8c5707ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:14.796106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e24fe8-7a98-44c3-b4f2-fd8b8c5707ea None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:14.837683 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:14.913747 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16775424-342c-4e6b-939d-fb2586f4a23d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:14.914284 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775424-342c-4e6b-939d-fb2586f4a23d service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:14.914602 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775424-342c-4e6b-939d-fb2586f4a23d service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:14.914870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775424-342c-4e6b-939d-fb2586f4a23d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:14.916484 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-16775424-342c-4e6b-939d-fb2586f4a23d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:15.012637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16775424-342c-4e6b-939d-fb2586f4a23d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:15.016247 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16775424-342c-4e6b-939d-fb2586f4a23d service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:15.044688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 716/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:16.167192 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb12b9-996b-419e-9e64-d96ab46a9d00 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:16.167670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb12b9-996b-419e-9e64-d96ab46a9d00 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:16.167783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb12b9-996b-419e-9e64-d96ab46a9d00 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:16.169770 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 16:25:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:16.178188 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b6a53-3a05-41ff-adb8-5cd8a6f9ec91 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:16.178638 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b6a53-3a05-41ff-adb8-5cd8a6f9ec91 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:16.178795 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b6a53-3a05-41ff-adb8-5cd8a6f9ec91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:16.237656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:16.262078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-190b6a53-3a05-41ff-adb8-5cd8a6f9ec91 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:16.263339 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-190b6a53-3a05-41ff-adb8-5cd8a6f9ec91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:16.365929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 16:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:16.515435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb12b9-996b-419e-9e64-d96ab46a9d00 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:16.517859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-19cb12b9-996b-419e-9e64-d96ab46a9d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:16.681077 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb12b9-996b-419e-9e64-d96ab46a9d00 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:16.695271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:16.803414 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1076249-cf9a-4591-88c5-3b88b58796cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:16.804024 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1076249-cf9a-4591-88c5-3b88b58796cb service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:16.804296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1076249-cf9a-4591-88c5-3b88b58796cb service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:16.804553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1076249-cf9a-4591-88c5-3b88b58796cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:16.806532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d1076249-cf9a-4591-88c5-3b88b58796cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:16.904851 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1076249-cf9a-4591-88c5-3b88b58796cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:16.906441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1076249-cf9a-4591-88c5-3b88b58796cb service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:16.927170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 705/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:18.035584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef97fa-f9c2-4827-8228-fba970cfcd73 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:18.036126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef97fa-f9c2-4827-8228-fba970cfcd73 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:18.036126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef97fa-f9c2-4827-8228-fba970cfcd73 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:18.038144 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 16:25:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:18.047734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d236f2-1730-4203-81d2-de297797ab9d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:18.048244 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d236f2-1730-4203-81d2-de297797ab9d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:18.048244 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d236f2-1730-4203-81d2-de297797ab9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:18.075003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:18.153637 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d9d236f2-1730-4203-81d2-de297797ab9d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:18.154902 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d9d236f2-1730-4203-81d2-de297797ab9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:18.279805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 16:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:25:18.439945 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef97fa-f9c2-4827-8228-fba970cfcd73 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:18.441529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b4ef97fa-f9c2-4827-8228-fba970cfcd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:18.567381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:18.603823 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef97fa-f9c2-4827-8228-fba970cfcd73 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:18.719204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979487e-cf52-4027-9bdc-32f4623bc7c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:18.719834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979487e-cf52-4027-9bdc-32f4623bc7c1 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:18.720141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979487e-cf52-4027-9bdc-32f4623bc7c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:18.720415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979487e-cf52-4027-9bdc-32f4623bc7c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:18.722160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-9979487e-cf52-4027-9bdc-32f4623bc7c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:18.797881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:18.812258 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979487e-cf52-4027-9bdc-32f4623bc7c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:18.813927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979487e-cf52-4027-9bdc-32f4623bc7c1 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:18.833572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 719/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:25:19.944093 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3d4dc-bb55-49f8-98ac-c68539498546 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:19.944669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3d4dc-bb55-49f8-98ac-c68539498546 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:19.944669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3d4dc-bb55-49f8-98ac-c68539498546 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:19.946785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 16:25:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:19.956333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7208c-55d5-48bf-90e9-fc3fdb80ad7d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:19.956798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7208c-55d5-48bf-90e9-fc3fdb80ad7d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:19.956890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7208c-55d5-48bf-90e9-fc3fdb80ad7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:20.035331 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-6fe7208c-55d5-48bf-90e9-fc3fdb80ad7d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:20.036547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-6fe7208c-55d5-48bf-90e9-fc3fdb80ad7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:20.139322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:25:20.288536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3d4dc-bb55-49f8-98ac-c68539498546 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:20.289954 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8cd3d4dc-bb55-49f8-98ac-c68539498546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:20.378671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:20.447230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3d4dc-bb55-49f8-98ac-c68539498546 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:20.566968 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365dbe3-c2e3-4ee3-a258-ba72f6683654 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:20.567490 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365dbe3-c2e3-4ee3-a258-ba72f6683654 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:20.567807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365dbe3-c2e3-4ee3-a258-ba72f6683654 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:20.568269 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365dbe3-c2e3-4ee3-a258-ba72f6683654 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:20.572248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0365dbe3-c2e3-4ee3-a258-ba72f6683654 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:20.626262 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:20.670233 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365dbe3-c2e3-4ee3-a258-ba72f6683654 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:20.671874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365dbe3-c2e3-4ee3-a258-ba72f6683654 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:20.691161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 708/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:21.743306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7a138-36eb-4a39-a540-2524d0b54537 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:21.744240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7a138-36eb-4a39-a540-2524d0b54537 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:21.744240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7a138-36eb-4a39-a540-2524d0b54537 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:21.746347 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 16:25:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:21.753640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac22f53-02de-4aa1-a04b-7ae65fe99c18 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:21.754094 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac22f53-02de-4aa1-a04b-7ae65fe99c18 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:21.754208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac22f53-02de-4aa1-a04b-7ae65fe99c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:21.834213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0ac22f53-02de-4aa1-a04b-7ae65fe99c18 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:21.835448 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0ac22f53-02de-4aa1-a04b-7ae65fe99c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:21.933913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 16:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:22.083006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7a138-36eb-4a39-a540-2524d0b54537 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:22.084536 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ecd7a138-36eb-4a39-a540-2524d0b54537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:22.134382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:22.237636 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7a138-36eb-4a39-a540-2524d0b54537 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:22.364157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27aea89-3897-4fe3-b579-aaedb60598c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:22.364784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27aea89-3897-4fe3-b579-aaedb60598c1 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:22.365062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27aea89-3897-4fe3-b579-aaedb60598c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:22.365311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27aea89-3897-4fe3-b579-aaedb60598c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:22.366948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e27aea89-3897-4fe3-b579-aaedb60598c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:22.374246 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:22.464974 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27aea89-3897-4fe3-b579-aaedb60598c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:22.466622 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27aea89-3897-4fe3-b579-aaedb60598c1 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:22.485301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 722/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:23.524496 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2b613-8788-433f-9489-5bfab1f7cec7 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:23.525119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2b613-8788-433f-9489-5bfab1f7cec7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:23.525119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2b613-8788-433f-9489-5bfab1f7cec7 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:23.526552 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 16:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:23.533754 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e7b4b-c503-4680-8f23-c06eb6db1c9f None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:23.534218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e7b4b-c503-4680-8f23-c06eb6db1c9f None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:23.534364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e7b4b-c503-4680-8f23-c06eb6db1c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:23.618782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f02e7b4b-c503-4680-8f23-c06eb6db1c9f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:23.620008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f02e7b4b-c503-4680-8f23-c06eb6db1c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:23.693887 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:23.723897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 16:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:23.875802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2b613-8788-433f-9489-5bfab1f7cec7 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:23.878183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-82b2b613-8788-433f-9489-5bfab1f7cec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:23.904913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:24.044360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2b613-8788-433f-9489-5bfab1f7cec7 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:24.135258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:24.161624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781c8c1-4dad-4d64-980e-2466a7a18e2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:24.162281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c8c1-4dad-4d64-980e-2466a7a18e2b service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:24.162574 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c8c1-4dad-4d64-980e-2466a7a18e2b service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:24.162840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c8c1-4dad-4d64-980e-2466a7a18e2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:24.164422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5781c8c1-4dad-4d64-980e-2466a7a18e2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:24.256512 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781c8c1-4dad-4d64-980e-2466a7a18e2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:24.258094 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781c8c1-4dad-4d64-980e-2466a7a18e2b service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:24.276449 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 711/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:25.326081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650b52d-80f8-4cdd-a9f8-67699b73bc9e None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:25.326657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650b52d-80f8-4cdd-a9f8-67699b73bc9e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:25.326657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650b52d-80f8-4cdd-a9f8-67699b73bc9e None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:25.327918 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 16:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:25.335197 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b02a6-3461-415a-93d8-4a2cf13d66f0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:25.335644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b02a6-3461-415a-93d8-4a2cf13d66f0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:25.335762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b02a6-3461-415a-93d8-4a2cf13d66f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:25.421141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a08b02a6-3461-415a-93d8-4a2cf13d66f0 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:25.422341 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a08b02a6-3461-415a-93d8-4a2cf13d66f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:25.464836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:25.530018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 16:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:25.678637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650b52d-80f8-4cdd-a9f8-67699b73bc9e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:25.680528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f650b52d-80f8-4cdd-a9f8-67699b73bc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:25.839145 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650b52d-80f8-4cdd-a9f8-67699b73bc9e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:25.893975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:25.953605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c067d8-2fc3-4351-9fa4-beacf40c4ce5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:25.954138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c067d8-2fc3-4351-9fa4-beacf40c4ce5 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:25.954409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c067d8-2fc3-4351-9fa4-beacf40c4ce5 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:25.954709 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c067d8-2fc3-4351-9fa4-beacf40c4ce5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:25.956314 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b7c067d8-2fc3-4351-9fa4-beacf40c4ce5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:26.046061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c067d8-2fc3-4351-9fa4-beacf40c4ce5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:26.047685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c067d8-2fc3-4351-9fa4-beacf40c4ce5 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:26.066277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 725/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:27.126261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac88f5-5c6f-4d4c-9af9-d6d9c09f315d None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:27.126795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac88f5-5c6f-4d4c-9af9-d6d9c09f315d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:27.126795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac88f5-5c6f-4d4c-9af9-d6d9c09f315d None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:27.128622 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 16:25:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:27.136184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b1d6d-3fe3-4b71-b80f-536511113223 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:27.136624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b1d6d-3fe3-4b71-b80f-536511113223 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:27.136767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b1d6d-3fe3-4b71-b80f-536511113223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:27.215667 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f22b1d6d-3fe3-4b71-b80f-536511113223 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:27.216864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f22b1d6d-3fe3-4b71-b80f-536511113223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:27.218977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:27.319088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 16:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:27.474052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac88f5-5c6f-4d4c-9af9-d6d9c09f315d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:27.475517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3fac88f5-5c6f-4d4c-9af9-d6d9c09f315d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:27.630645 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac88f5-5c6f-4d4c-9af9-d6d9c09f315d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:27.659079 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:27.747242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6076c21-fbde-4e6a-81ef-fca050734c82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:27.747766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6076c21-fbde-4e6a-81ef-fca050734c82 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:27.748037 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6076c21-fbde-4e6a-81ef-fca050734c82 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:27.748297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6076c21-fbde-4e6a-81ef-fca050734c82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:27.749860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e6076c21-fbde-4e6a-81ef-fca050734c82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:27.842788 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6076c21-fbde-4e6a-81ef-fca050734c82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:27.844416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6076c21-fbde-4e6a-81ef-fca050734c82 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:27.862810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 714/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:29.891807 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fea226-238e-4863-963f-b248e7369b37 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:29.892432 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fea226-238e-4863-963f-b248e7369b37 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:29.892432 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fea226-238e-4863-963f-b248e7369b37 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:29.893691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:29.900233 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4072569-104c-4414-9f24-271aeb2576a8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:29.900993 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4072569-104c-4414-9f24-271aeb2576a8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:29.901183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4072569-104c-4414-9f24-271aeb2576a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:29.951342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:29.978903 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a4072569-104c-4414-9f24-271aeb2576a8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:29.980099 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a4072569-104c-4414-9f24-271aeb2576a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:30.082766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:30.365412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fea226-238e-4863-963f-b248e7369b37 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:30.367824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-53fea226-238e-4863-963f-b248e7369b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:30.493791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fea226-238e-4863-963f-b248e7369b37 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:30.528193 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:30.572561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa4a8a-438d-46d4-b02f-cabe4239050d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:30.573169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa4a8a-438d-46d4-b02f-cabe4239050d service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:30.573474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa4a8a-438d-46d4-b02f-cabe4239050d service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:30.573740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa4a8a-438d-46d4-b02f-cabe4239050d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:30.575376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-31aa4a8a-438d-46d4-b02f-cabe4239050d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:30.816029 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa4a8a-438d-46d4-b02f-cabe4239050d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:30.817551 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa4a8a-438d-46d4-b02f-cabe4239050d service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:30.835999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 728/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:33.175614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d92bca-0f3e-4a6b-900e-439972257fd3 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:33.176240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d92bca-0f3e-4a6b-900e-439972257fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:33.176240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d92bca-0f3e-4a6b-900e-439972257fd3 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:33.177483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:33.184290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa7f15-04e9-469d-89e2-7327b26dff54 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:33.184924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa7f15-04e9-469d-89e2-7327b26dff54 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:33.185069 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa7f15-04e9-469d-89e2-7327b26dff54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:33.268567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-73aa7f15-04e9-469d-89e2-7327b26dff54 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:33.269793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-73aa7f15-04e9-469d-89e2-7327b26dff54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:33.283239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:33.371713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:33.663190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d92bca-0f3e-4a6b-900e-439972257fd3 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:33.664717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-77d92bca-0f3e-4a6b-900e-439972257fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:33.792590 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d92bca-0f3e-4a6b-900e-439972257fd3 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:33.806648 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:33.877558 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872455d-191e-4cff-bf47-d71af348d6d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:33.878164 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872455d-191e-4cff-bf47-d71af348d6d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:33.878471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872455d-191e-4cff-bf47-d71af348d6d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:33.878774 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872455d-191e-4cff-bf47-d71af348d6d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:33.880402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4872455d-191e-4cff-bf47-d71af348d6d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:33.993990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872455d-191e-4cff-bf47-d71af348d6d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:33.995450 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872455d-191e-4cff-bf47-d71af348d6d9 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:34.014449 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 717/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:25:36.462228 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204b82e-6038-4bf9-b73a-712608c6e5d1 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:36.462869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204b82e-6038-4bf9-b73a-712608c6e5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:36.462869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204b82e-6038-4bf9-b73a-712608c6e5d1 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:36.464208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:36.470073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c1256-e8ac-4b72-9dd4-16029d923c80 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:36.470551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c1256-e8ac-4b72-9dd4-16029d923c80 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:36.470668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c1256-e8ac-4b72-9dd4-16029d923c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:36.524377 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:36.544297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ab7c1256-e8ac-4b72-9dd4-16029d923c80 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:36.545498 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ab7c1256-e8ac-4b72-9dd4-16029d923c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:36.648908 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:25:36.933380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204b82e-6038-4bf9-b73a-712608c6e5d1 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:36.934972 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1204b82e-6038-4bf9-b73a-712608c6e5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:37.048312 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204b82e-6038-4bf9-b73a-712608c6e5d1 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:37.101862 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:37.136524 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c98df2-1b07-4991-b5a8-5ad32992b53a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:37.137059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c98df2-1b07-4991-b5a8-5ad32992b53a service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:37.137329 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c98df2-1b07-4991-b5a8-5ad32992b53a service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:37.137588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c98df2-1b07-4991-b5a8-5ad32992b53a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:37.139205 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e3c98df2-1b07-4991-b5a8-5ad32992b53a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:37.276875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c98df2-1b07-4991-b5a8-5ad32992b53a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:37.278345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c98df2-1b07-4991-b5a8-5ad32992b53a service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:37.297530 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 731/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:39.551724 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e9d5e-2e02-4391-ab17-bfdcd916054e None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:39.552371 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e9d5e-2e02-4391-ab17-bfdcd916054e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:39.552371 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e9d5e-2e02-4391-ab17-bfdcd916054e None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:39.553626 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:39.560556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64e626-e482-4cf6-8a8b-2adfb6d18f03 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:39.561248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64e626-e482-4cf6-8a8b-2adfb6d18f03 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:39.561369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64e626-e482-4cf6-8a8b-2adfb6d18f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:39.634458 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-3c64e626-e482-4cf6-8a8b-2adfb6d18f03 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:39.635711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-3c64e626-e482-4cf6-8a8b-2adfb6d18f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:39.658656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:39.749601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:40.030784 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e9d5e-2e02-4391-ab17-bfdcd916054e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:40.032317 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-361e9d5e-2e02-4391-ab17-bfdcd916054e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:40.143515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e9d5e-2e02-4391-ab17-bfdcd916054e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:40.163783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:40.224316 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1df3e-4444-4b0e-95a0-60f02f96461d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:40.224916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1df3e-4444-4b0e-95a0-60f02f96461d service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:40.225187 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1df3e-4444-4b0e-95a0-60f02f96461d service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:40.225717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1df3e-4444-4b0e-95a0-60f02f96461d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:40.227345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-03a1df3e-4444-4b0e-95a0-60f02f96461d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:40.338953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1df3e-4444-4b0e-95a0-60f02f96461d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:40.340307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1df3e-4444-4b0e-95a0-60f02f96461d service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:40.358324 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 720/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:42.565110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bafe04-0d5b-473c-aafd-3474351dbcde None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:42.566350 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bafe04-0d5b-473c-aafd-3474351dbcde None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:42.566350 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bafe04-0d5b-473c-aafd-3474351dbcde None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:42.567084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:42.572679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a902a-b773-4fe6-8b76-e2bdf1518a43 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:42.573144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a902a-b773-4fe6-8b76-e2bdf1518a43 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:42.573240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a902a-b773-4fe6-8b76-e2bdf1518a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:42.638466 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:42.641685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-6a2a902a-b773-4fe6-8b76-e2bdf1518a43 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:42.643707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6a2a902a-b773-4fe6-8b76-e2bdf1518a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:42.745408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:43.023826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bafe04-0d5b-473c-aafd-3474351dbcde None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:43.025377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-86bafe04-0d5b-473c-aafd-3474351dbcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:43.136456 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bafe04-0d5b-473c-aafd-3474351dbcde None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:43.187514 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:43.213241 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2f2dd-89e7-4327-83a9-820d20ca3454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:43.213858 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2f2dd-89e7-4327-83a9-820d20ca3454 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:43.214133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2f2dd-89e7-4327-83a9-820d20ca3454 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:43.214400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2f2dd-89e7-4327-83a9-820d20ca3454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:43.216043 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f0f2f2dd-89e7-4327-83a9-820d20ca3454 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:43.329427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2f2dd-89e7-4327-83a9-820d20ca3454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:43.330936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2f2dd-89e7-4327-83a9-820d20ca3454 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:43.350753 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 734/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:25:45.544343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c030d85-96ad-4281-b362-b71650292a21 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:45.544915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c030d85-96ad-4281-b362-b71650292a21 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:45.544915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c030d85-96ad-4281-b362-b71650292a21 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:45.546260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:45.552864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ca50c-b6af-42e4-b111-7727328d0704 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:45.553306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ca50c-b6af-42e4-b111-7727328d0704 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:45.553747 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ca50c-b6af-42e4-b111-7727328d0704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:45.628889 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-601ca50c-b6af-42e4-b111-7727328d0704 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:45.630146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-601ca50c-b6af-42e4-b111-7727328d0704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:45.660692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:45.731862 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:46.019931 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c030d85-96ad-4281-b362-b71650292a21 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:46.021480 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4c030d85-96ad-4281-b362-b71650292a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:46.132711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c030d85-96ad-4281-b362-b71650292a21 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:46.163375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:46.211145 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fa2e8-b272-4238-8955-135f2df40dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:46.211754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fa2e8-b272-4238-8955-135f2df40dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:46.212032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fa2e8-b272-4238-8955-135f2df40dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:46.212297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fa2e8-b272-4238-8955-135f2df40dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:46.213870 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-586fa2e8-b272-4238-8955-135f2df40dc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:46.325958 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586fa2e8-b272-4238-8955-135f2df40dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:46.327341 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586fa2e8-b272-4238-8955-135f2df40dc8 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:46.345789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 723/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:48.563649 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf54c10-51e9-4d91-8bff-e149c71db59c None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:48.564260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf54c10-51e9-4d91-8bff-e149c71db59c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:48.564260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf54c10-51e9-4d91-8bff-e149c71db59c None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:48.565536 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:48.572149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c7ac0-adfe-46eb-9b8c-ae18c6845aa1 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:48.572588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c7ac0-adfe-46eb-9b8c-ae18c6845aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:48.572683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c7ac0-adfe-46eb-9b8c-ae18c6845aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:48.652365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-358c7ac0-adfe-46eb-9b8c-ae18c6845aa1 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:48.653482 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-358c7ac0-adfe-46eb-9b8c-ae18c6845aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:48.662282 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:48.767995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:49.046534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf54c10-51e9-4d91-8bff-e149c71db59c None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:49.048160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-adf54c10-51e9-4d91-8bff-e149c71db59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:49.165906 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf54c10-51e9-4d91-8bff-e149c71db59c None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:49.222135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:49.241999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462aca8-af56-4073-9e1d-937be97f6185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:49.242655 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462aca8-af56-4073-9e1d-937be97f6185 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:49.242932 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462aca8-af56-4073-9e1d-937be97f6185 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:49.243201 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462aca8-af56-4073-9e1d-937be97f6185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:49.244802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b462aca8-af56-4073-9e1d-937be97f6185 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:49.399042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462aca8-af56-4073-9e1d-937be97f6185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:49.400468 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462aca8-af56-4073-9e1d-937be97f6185 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:49.418691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 737/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:51.644196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d136ce8-0b7b-4a46-a807-a848cfe87672 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:51.645257 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d136ce8-0b7b-4a46-a807-a848cfe87672 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:51.645257 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d136ce8-0b7b-4a46-a807-a848cfe87672 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:51.647486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:25:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:51.653014 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c941c-0317-4e5d-9eac-1c058b163727 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:51.653500 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c941c-0317-4e5d-9eac-1c058b163727 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:51.653500 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c941c-0317-4e5d-9eac-1c058b163727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:51.731862 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-653c941c-0317-4e5d-9eac-1c058b163727 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:51.733073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-653c941c-0317-4e5d-9eac-1c058b163727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:51.773635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:51.835503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:52.119436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d136ce8-0b7b-4a46-a807-a848cfe87672 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:52.121747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9d136ce8-0b7b-4a46-a807-a848cfe87672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:52.236359 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d136ce8-0b7b-4a46-a807-a848cfe87672 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:52.276048 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:52.316339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee49dbc-ea9d-4bdb-8d03-dc33e18bfdbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:52.317052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee49dbc-ea9d-4bdb-8d03-dc33e18bfdbb service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:52.317328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee49dbc-ea9d-4bdb-8d03-dc33e18bfdbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:52.317598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee49dbc-ea9d-4bdb-8d03-dc33e18bfdbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:52.319231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-aee49dbc-ea9d-4bdb-8d03-dc33e18bfdbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:52.442655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee49dbc-ea9d-4bdb-8d03-dc33e18bfdbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:52.444098 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee49dbc-ea9d-4bdb-8d03-dc33e18bfdbb service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:52.463385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 726/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:54.627295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb84c85-1e7d-4546-b047-4b1ac65a71fe None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:54.628168 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb84c85-1e7d-4546-b047-4b1ac65a71fe None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:54.628337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb84c85-1e7d-4546-b047-4b1ac65a71fe None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:54.630143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:25:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:54.638130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8024c7-c10a-480a-a6e8-e77ea615fb71 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:54.638616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8024c7-c10a-480a-a6e8-e77ea615fb71 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:54.638767 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8024c7-c10a-480a-a6e8-e77ea615fb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:54.706747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5e8024c7-c10a-480a-a6e8-e77ea615fb71 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:54.707973 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5e8024c7-c10a-480a-a6e8-e77ea615fb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:54.726007 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:54.806013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:55.087694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb84c85-1e7d-4546-b047-4b1ac65a71fe None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:55.089317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2fb84c85-1e7d-4546-b047-4b1ac65a71fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:55.204198 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb84c85-1e7d-4546-b047-4b1ac65a71fe None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:55.273909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:55.280996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edbd4b-5741-447b-9435-bc4e7c680f47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:55.281605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edbd4b-5741-447b-9435-bc4e7c680f47 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:55.281838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edbd4b-5741-447b-9435-bc4e7c680f47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:55.282133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edbd4b-5741-447b-9435-bc4e7c680f47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:55.283743 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f3edbd4b-5741-447b-9435-bc4e7c680f47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:55.403436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edbd4b-5741-447b-9435-bc4e7c680f47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:55.404897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edbd4b-5741-447b-9435-bc4e7c680f47 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:55.422793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 740/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:25:57.572942 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b82450-8212-403c-871b-5177fc3737ae None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:57.573539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b82450-8212-403c-871b-5177fc3737ae None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:57.573539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b82450-8212-403c-871b-5177fc3737ae None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:57.575671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:25:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:25:57.584835 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2dcc-d040-4373-b64c-c5eadc583d89 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:57.585239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2dcc-d040-4373-b64c-c5eadc583d89 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:57.585443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2dcc-d040-4373-b64c-c5eadc583d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:57.659016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-7ffa2dcc-d040-4373-b64c-c5eadc583d89 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:25:57.660217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7ffa2dcc-d040-4373-b64c-c5eadc583d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:57.723320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:57.776279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:25:58.056660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b82450-8212-403c-871b-5177fc3737ae None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:25:58.058941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-10b82450-8212-403c-871b-5177fc3737ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:58.174767 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b82450-8212-403c-871b-5177fc3737ae None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:25:58.225231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:25:58.252976 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ff2fe-4153-4c1d-9f54-1958e3cf5bed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:25:58.253579 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ff2fe-4153-4c1d-9f54-1958e3cf5bed service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:25:58.253836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ff2fe-4153-4c1d-9f54-1958e3cf5bed service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:25:58.254114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ff2fe-4153-4c1d-9f54-1958e3cf5bed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:25:58.255832 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d83ff2fe-4153-4c1d-9f54-1958e3cf5bed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:25:58.370334 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ff2fe-4153-4c1d-9f54-1958e3cf5bed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:25:58.371809 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ff2fe-4153-4c1d-9f54-1958e3cf5bed service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:25:58.391060 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 729/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:25:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:26:07.774685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=22 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:08.388182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=66 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:18.180839 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7ab6b-d5fe-42f2-957a-2b09eaa4d9db None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:18.181774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7ab6b-d5fe-42f2-957a-2b09eaa4d9db None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:18.181774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7ab6b-d5fe-42f2-957a-2b09eaa4d9db None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:18.184013 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 742/1471] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:26:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:26:18.190476 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d528a-dceb-4a26-a4da-c3e194c57728 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:18.191240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d528a-dceb-4a26-a4da-c3e194c57728 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:18.191345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d528a-dceb-4a26-a4da-c3e194c57728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:18.268506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-154d528a-dceb-4a26-a4da-c3e194c57728 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:26:18.269720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-154d528a-dceb-4a26-a4da-c3e194c57728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:18.377949 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 730/1472] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:26:18.675270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74576c6-8d84-4dc9-90f2-4afc8dd85b6b None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:18.675546 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74576c6-8d84-4dc9-90f2-4afc8dd85b6b None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:18.675813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74576c6-8d84-4dc9-90f2-4afc8dd85b6b None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:18.677194 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 743/1473] 10.222.0.23 () {54 vars in 966 bytes} [Fri Sep 9 16:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:26:18.684536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948eb53-8ef0-43fe-a066-e34fbefa1753 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:18.684816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948eb53-8ef0-43fe-a066-e34fbefa1753 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:18.685092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948eb53-8ef0-43fe-a066-e34fbefa1753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:18.764516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b948eb53-8ef0-43fe-a066-e34fbefa1753 None None] MFA Rules not processed for user `62a2fb70c9ae4f4b81ac4de513b868cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:26:18.765754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b948eb53-8ef0-43fe-a066-e34fbefa1753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:18.787717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:18.856225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 731/1474] 10.222.0.23 () {58 vars in 1063 bytes} [Fri Sep 9 16:26:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:26:18.864578 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74576c6-8d84-4dc9-90f2-4afc8dd85b6b None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:26:18.866142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d74576c6-8d84-4dc9-90f2-4afc8dd85b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:18.978291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74576c6-8d84-4dc9-90f2-4afc8dd85b6b None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:26:19.066849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c4141-2b5f-4fcb-beff-c2a0983134ee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a2fb70c9ae4f4b81ac4de513b868cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:26:19.067461 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c4141-2b5f-4fcb-beff-c2a0983134ee service glance] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:19.067738 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c4141-2b5f-4fcb-beff-c2a0983134ee service glance] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:19.068010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c4141-2b5f-4fcb-beff-c2a0983134ee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:19.069613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5b8c4141-2b5f-4fcb-beff-c2a0983134ee service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:19.124415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:19.180909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c4141-2b5f-4fcb-beff-c2a0983134ee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:26:19.182521 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c4141-2b5f-4fcb-beff-c2a0983134ee service glance] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:26:19.201376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 744/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:26:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:26:21.418929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a330d0f-00db-4498-ba0a-5d36efb0c61b None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:21.419503 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a330d0f-00db-4498-ba0a-5d36efb0c61b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:21.419503 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a330d0f-00db-4498-ba0a-5d36efb0c61b None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:21.420831 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 732/1476] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:26:21.427667 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858037a4-defb-4d26-bdf2-3f8a9d98e78d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:21.428115 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858037a4-defb-4d26-bdf2-3f8a9d98e78d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:21.428232 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858037a4-defb-4d26-bdf2-3f8a9d98e78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:21.510176 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-858037a4-defb-4d26-bdf2-3f8a9d98e78d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:26:21.511424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-858037a4-defb-4d26-bdf2-3f8a9d98e78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:21.617920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 745/1477] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:26:21.887023 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a330d0f-00db-4498-ba0a-5d36efb0c61b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:26:21.888573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0a330d0f-00db-4498-ba0a-5d36efb0c61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:22.024038 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a330d0f-00db-4498-ba0a-5d36efb0c61b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:26:22.082257 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:22.125493 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143e57a-6021-406b-a96a-e7b7386a036f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:26:22.126110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143e57a-6021-406b-a96a-e7b7386a036f admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:22.126573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143e57a-6021-406b-a96a-e7b7386a036f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:22.127051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143e57a-6021-406b-a96a-e7b7386a036f admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:22.128189 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143e57a-6021-406b-a96a-e7b7386a036f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:26:22.131329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-f143e57a-6021-406b-a96a-e7b7386a036f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:26:22.131329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Sep 09 16:26:22.131329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: Sep 09 16:26:22.132730 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143e57a-6021-406b-a96a-e7b7386a036f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:26:22.155377 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 733/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Sep 9 16:26:21 2022] GET /identity/v3/projects => generated 2553 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:26:24.314533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08963b44-222d-49ff-ae58-417c788e3190 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:24.315134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08963b44-222d-49ff-ae58-417c788e3190 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:24.315134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08963b44-222d-49ff-ae58-417c788e3190 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:24.316504 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 746/1479] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:26:24.323215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a89626-f8c0-4b53-a56f-bed83b6d518c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:24.323683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a89626-f8c0-4b53-a56f-bed83b6d518c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:24.323929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a89626-f8c0-4b53-a56f-bed83b6d518c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:24.394449 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f9a89626-f8c0-4b53-a56f-bed83b6d518c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:26:24.395877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f9a89626-f8c0-4b53-a56f-bed83b6d518c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:24.497927 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 734/1480] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:26:24.788260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08963b44-222d-49ff-ae58-417c788e3190 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:26:24.789802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-08963b44-222d-49ff-ae58-417c788e3190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:24.800601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:24.945172 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08963b44-222d-49ff-ae58-417c788e3190 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:26:25.033912 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:25.067919 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e65ba-e094-4ebb-ac4f-20adea260b24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:26:25.068525 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e65ba-e094-4ebb-ac4f-20adea260b24 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:25.068819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e65ba-e094-4ebb-ac4f-20adea260b24 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:25.069054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e65ba-e094-4ebb-ac4f-20adea260b24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:25.070682 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0f2e65ba-e094-4ebb-ac4f-20adea260b24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:25.171566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e65ba-e094-4ebb-ac4f-20adea260b24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:26:25.173185 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e65ba-e094-4ebb-ac4f-20adea260b24 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:26:25.192374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 747/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:26:27.520231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad203bec-ada9-4180-83f2-82912c665af8 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:27.521163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad203bec-ada9-4180-83f2-82912c665af8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:27.521163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad203bec-ada9-4180-83f2-82912c665af8 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:27.523722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 735/1482] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:26:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:26:27.529366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e299a-eae5-4634-aa79-e44f63ac6908 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:27.529808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e299a-eae5-4634-aa79-e44f63ac6908 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:27.529911 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e299a-eae5-4634-aa79-e44f63ac6908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:27.606440 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-305e299a-eae5-4634-aa79-e44f63ac6908 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:26:27.607682 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-305e299a-eae5-4634-aa79-e44f63ac6908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:27.655775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:27.709475 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 748/1483] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:26:27.993703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad203bec-ada9-4180-83f2-82912c665af8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:26:27.995301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ad203bec-ada9-4180-83f2-82912c665af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:28.019582 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:28.103782 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad203bec-ada9-4180-83f2-82912c665af8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:26:28.184644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f15ab-8928-4805-9426-9f4b5fce964e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:26:28.185262 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f15ab-8928-4805-9426-9f4b5fce964e service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:28.185514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f15ab-8928-4805-9426-9f4b5fce964e service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:28.185791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f15ab-8928-4805-9426-9f4b5fce964e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:28.187429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8c2f15ab-8928-4805-9426-9f4b5fce964e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:28.265360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:28.305603 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f15ab-8928-4805-9426-9f4b5fce964e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:26:28.307102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f15ab-8928-4805-9426-9f4b5fce964e service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:26:28.324735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 736/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:26:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:26:30.510661 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f946a7-be1b-44c5-8ae6-981f26a4ee16 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:30.511266 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f946a7-be1b-44c5-8ae6-981f26a4ee16 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:30.511266 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f946a7-be1b-44c5-8ae6-981f26a4ee16 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:30.512553 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 749/1485] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:26:30.518983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8feac2c-2692-4e39-ac63-bda0856c0b0b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:30.519412 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8feac2c-2692-4e39-ac63-bda0856c0b0b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:30.519519 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8feac2c-2692-4e39-ac63-bda0856c0b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:30.595293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c8feac2c-2692-4e39-ac63-bda0856c0b0b None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:26:30.596516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c8feac2c-2692-4e39-ac63-bda0856c0b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:30.697864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 737/1486] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:26:30.992317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f946a7-be1b-44c5-8ae6-981f26a4ee16 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:26:30.993906 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-69f946a7-be1b-44c5-8ae6-981f26a4ee16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:31.104011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f946a7-be1b-44c5-8ae6-981f26a4ee16 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:26:31.191383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:31.193340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e56b92-3838-4b8a-b62f-3a814c95b9ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:26:31.193975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e56b92-3838-4b8a-b62f-3a814c95b9ea service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:31.194269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e56b92-3838-4b8a-b62f-3a814c95b9ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:31.194774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e56b92-3838-4b8a-b62f-3a814c95b9ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:31.196895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-89e56b92-3838-4b8a-b62f-3a814c95b9ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:31.311593 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e56b92-3838-4b8a-b62f-3a814c95b9ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:26:31.313054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e56b92-3838-4b8a-b62f-3a814c95b9ea service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:26:31.331049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 750/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:26:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:26:33.564210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cf804-0d19-4756-b7f2-f7f5198724bc None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:33.564783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cf804-0d19-4756-b7f2-f7f5198724bc None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:33.564783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cf804-0d19-4756-b7f2-f7f5198724bc None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:33.566128 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 738/1488] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:26:33.572685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516aaac-727d-4bd4-a5dc-555ba1546130 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:33.573372 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516aaac-727d-4bd4-a5dc-555ba1546130 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:33.573642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516aaac-727d-4bd4-a5dc-555ba1546130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:33.652098 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-c516aaac-727d-4bd4-a5dc-555ba1546130 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:26:33.653309 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c516aaac-727d-4bd4-a5dc-555ba1546130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:33.709034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:33.760597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 751/1489] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:26:34.052527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5cf804-0d19-4756-b7f2-f7f5198724bc None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:26:34.054084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2a5cf804-0d19-4756-b7f2-f7f5198724bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:34.094435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:34.165645 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5cf804-0d19-4756-b7f2-f7f5198724bc None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:26:34.244640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5d3eb-9ccb-468b-b915-67b22c6ea45a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:26:34.245254 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5d3eb-9ccb-468b-b915-67b22c6ea45a service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:34.245549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5d3eb-9ccb-468b-b915-67b22c6ea45a service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:34.245804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5d3eb-9ccb-468b-b915-67b22c6ea45a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:34.247456 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-74b5d3eb-9ccb-468b-b915-67b22c6ea45a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:34.342802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:34.371985 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5d3eb-9ccb-468b-b915-67b22c6ea45a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:26:34.373440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5d3eb-9ccb-468b-b915-67b22c6ea45a service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:26:34.396183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 739/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:26:36.748132 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea31719-f255-4c1b-b156-219fdf03c8f2 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:36.749054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea31719-f255-4c1b-b156-219fdf03c8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:36.749054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea31719-f255-4c1b-b156-219fdf03c8f2 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:36.751240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 752/1491] 10.222.0.23 () {54 vars in 927 bytes} [Fri Sep 9 16:26:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:26:36.758743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad9f06-98f2-40a1-8f8a-2420a63b0c5b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:36.759158 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad9f06-98f2-40a1-8f8a-2420a63b0c5b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:36.759289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad9f06-98f2-40a1-8f8a-2420a63b0c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:36.828892 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-acad9f06-98f2-40a1-8f8a-2420a63b0c5b None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:26:36.830100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-acad9f06-98f2-40a1-8f8a-2420a63b0c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:36.934567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 740/1492] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Sep 9 16:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:26:37.233892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea31719-f255-4c1b-b156-219fdf03c8f2 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:26:37.235508 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-4ea31719-f255-4c1b-b156-219fdf03c8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:37.346423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea31719-f255-4c1b-b156-219fdf03c8f2 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:26:37.443886 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc9ae6-7f33-402a-bac1-691abcf5870c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:26:37.444618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc9ae6-7f33-402a-bac1-691abcf5870c service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:26:37.444897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc9ae6-7f33-402a-bac1-691abcf5870c service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:26:37.445166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc9ae6-7f33-402a-bac1-691abcf5870c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:26:37.447142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2cfc9ae6-7f33-402a-bac1-691abcf5870c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:26:37.456733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:37.560615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc9ae6-7f33-402a-bac1-691abcf5870c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:26:37.562036 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc9ae6-7f33-402a-bac1-691abcf5870c service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:26:37.579861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 753/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:26:46.932819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=82 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:26:47.577926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=50 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:28:16.312119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2093587-580a-4d73-9205-a3d35f389491 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:16.312798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2093587-580a-4d73-9205-a3d35f389491 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:16.312798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2093587-580a-4d73-9205-a3d35f389491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:16.397607 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b2093587-580a-4d73-9205-a3d35f389491 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:28:16.398847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b2093587-580a-4d73-9205-a3d35f389491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:28:16.508041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 741/1494] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:28:16.516539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3f3e8-3f31-4de3-b407-931069577213 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:16.517093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3f3e8-3f31-4de3-b407-931069577213 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:16.517093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3f3e8-3f31-4de3-b407-931069577213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:16.597202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-aed3f3e8-3f31-4de3-b407-931069577213 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:28:16.598437 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-aed3f3e8-3f31-4de3-b407-931069577213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:28:16.697178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 754/1495] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:28:16.702272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2093587-580a-4d73-9205-a3d35f389491 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:28:16.703918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b2093587-580a-4d73-9205-a3d35f389491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:28:16.718351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2093587-580a-4d73-9205-a3d35f389491 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:28:16.784612 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:28:16.819035 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6ccd6-163a-438d-9155-73394c2aefb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:28:16.819641 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6ccd6-163a-438d-9155-73394c2aefb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:16.819917 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6ccd6-163a-438d-9155-73394c2aefb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:16.820183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6ccd6-163a-438d-9155-73394c2aefb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:16.820823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6ccd6-163a-438d-9155-73394c2aefb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:28:16.821120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6ccd6-163a-438d-9155-73394c2aefb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:28:16.822140 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-dba6ccd6-163a-438d-9155-73394c2aefb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:28:16.822140 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Sep 09 16:28:16.822140 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: Sep 09 16:28:16.823059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6ccd6-163a-438d-9155-73394c2aefb1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:28:16.834963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 742/1496] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:28:16.843526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3f3e8-3f31-4de3-b407-931069577213 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:28:16.853833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3f3e8-3f31-4de3-b407-931069577213 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:28:16.922624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:28:16.961653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42784b-28db-4005-9f77-02e527238ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:28:16.962273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42784b-28db-4005-9f77-02e527238ffa admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:16.962593 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42784b-28db-4005-9f77-02e527238ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:16.962858 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42784b-28db-4005-9f77-02e527238ffa admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:16.963485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42784b-28db-4005-9f77-02e527238ffa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:28:16.965257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: WARNING py.warnings [None req-fe42784b-28db-4005-9f77-02e527238ffa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:28:16.965257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: warnings.warn(msg) Sep 09 16:28:16.965257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: Sep 09 16:28:16.966476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42784b-28db-4005-9f77-02e527238ffa admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:28:17.028462 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 755/1497] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:28:16 2022] POST /identity/v3/projects => generated 379 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:28:17.032988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6ccd6-163a-438d-9155-73394c2aefb1 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:28:17.034520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dba6ccd6-163a-438d-9155-73394c2aefb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:28:17.069908 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6ccd6-163a-438d-9155-73394c2aefb1 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:28:17.169471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce5a0a-a213-4eac-8e52-01bc05226514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:28:17.170008 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce5a0a-a213-4eac-8e52-01bc05226514 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:17.170286 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce5a0a-a213-4eac-8e52-01bc05226514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:17.170582 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce5a0a-a213-4eac-8e52-01bc05226514 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:17.171199 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce5a0a-a213-4eac-8e52-01bc05226514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:28:17.172318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-c6ce5a0a-a213-4eac-8e52-01bc05226514 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:28:17.172318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Sep 09 16:28:17.172318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: Sep 09 16:28:17.173565 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce5a0a-a213-4eac-8e52-01bc05226514 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:28:17.220073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 743/1498] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:28:17 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:28:17.224610 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42784b-28db-4005-9f77-02e527238ffa admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:28:17.226124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-fe42784b-28db-4005-9f77-02e527238ffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:28:17.289633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42784b-28db-4005-9f77-02e527238ffa admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:28:17.377457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:28:17.388927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afcc6c-2138-48f0-98d4-a9024e7d1f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:28:17.389784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afcc6c-2138-48f0-98d4-a9024e7d1f79 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:17.390125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afcc6c-2138-48f0-98d4-a9024e7d1f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:17.390460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afcc6c-2138-48f0-98d4-a9024e7d1f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:17.391207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afcc6c-2138-48f0-98d4-a9024e7d1f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:28:17.392492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: WARNING py.warnings [None req-12afcc6c-2138-48f0-98d4-a9024e7d1f79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:28:17.392492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: warnings.warn(msg) Sep 09 16:28:17.392492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: Sep 09 16:28:17.393380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afcc6c-2138-48f0-98d4-a9024e7d1f79 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:28:17.406020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 756/1499] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:28:17.410270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce5a0a-a213-4eac-8e52-01bc05226514 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:28:17.411820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c6ce5a0a-a213-4eac-8e52-01bc05226514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:28:17.423740 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce5a0a-a213-4eac-8e52-01bc05226514 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:28:17.449439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:28:17.523160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0068d6d-8e24-4714-9f64-09e51c55a4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:28:17.523771 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0068d6d-8e24-4714-9f64-09e51c55a4fc admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:17.524047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0068d6d-8e24-4714-9f64-09e51c55a4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:17.524316 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0068d6d-8e24-4714-9f64-09e51c55a4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:17.524962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0068d6d-8e24-4714-9f64-09e51c55a4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:28:17.525993 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-f0068d6d-8e24-4714-9f64-09e51c55a4fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:28:17.525993 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Sep 09 16:28:17.525993 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: Sep 09 16:28:17.526913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0068d6d-8e24-4714-9f64-09e51c55a4fc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:28:17.540023 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 744/1500] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:28:17.545044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afcc6c-2138-48f0-98d4-a9024e7d1f79 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:28:17.547139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-12afcc6c-2138-48f0-98d4-a9024e7d1f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:28:17.586219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afcc6c-2138-48f0-98d4-a9024e7d1f79 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:28:17.679678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cbfde-3d0d-4c61-92aa-744bd480d867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:28:17.680245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cbfde-3d0d-4c61-92aa-744bd480d867 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:17.680519 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cbfde-3d0d-4c61-92aa-744bd480d867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:17.680785 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cbfde-3d0d-4c61-92aa-744bd480d867 admin admin] PATH_INFO: `/v3/projects/a59a3d120687475e947c2c97ca538c25/users/b49e00847418493ebddba2a76a5b3b9d/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:17.709026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478cbfde-3d0d-4c61-92aa-744bd480d867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a59a3d120687475e947c2c97ca538c25, user_id=b49e00847418493ebddba2a76a5b3b9d, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:28:17.710526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: WARNING py.warnings [None req-478cbfde-3d0d-4c61-92aa-744bd480d867 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:28:17.710526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: warnings.warn(msg) Sep 09 16:28:17.710526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: Sep 09 16:28:17.713423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478cbfde-3d0d-4c61-92aa-744bd480d867 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:28:17.742955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 757/1501] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:28:17 2022] PUT /identity/v3/projects/a59a3d120687475e947c2c97ca538c25/users/b49e00847418493ebddba2a76a5b3b9d/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:28:17.755592 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefc04b-e3f7-44fa-a195-6ebdc9bda154 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:17.755894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefc04b-e3f7-44fa-a195-6ebdc9bda154 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:17.756154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefc04b-e3f7-44fa-a195-6ebdc9bda154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:17.823556 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-adefc04b-e3f7-44fa-a195-6ebdc9bda154 None None] MFA Rules not processed for user `b49e00847418493ebddba2a76a5b3b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:28:17.824754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-adefc04b-e3f7-44fa-a195-6ebdc9bda154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:28:17.918387 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 745/1502] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:28:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:28:17.932783 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7d88d-fd0c-47f1-8a57-3f4bd90acb39 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:17.933022 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7d88d-fd0c-47f1-8a57-3f4bd90acb39 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:17.933295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7d88d-fd0c-47f1-8a57-3f4bd90acb39 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:17.934692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 758/1503] 10.222.0.23 () {54 vars in 964 bytes} [Fri Sep 9 16:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:28:17.939856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51825965-5518-43cc-b7b8-6eba600a646b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:17.940138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51825965-5518-43cc-b7b8-6eba600a646b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:17.940407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51825965-5518-43cc-b7b8-6eba600a646b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:17.981232 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:28:18.014887 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-51825965-5518-43cc-b7b8-6eba600a646b None None] MFA Rules not processed for user `95b665b9e27144b5bd684ece7ff465e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:28:18.016149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-51825965-5518-43cc-b7b8-6eba600a646b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:28:18.105163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 746/1504] 10.222.0.23 () {58 vars in 1061 bytes} [Fri Sep 9 16:28:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:28:18.113326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7d88d-fd0c-47f1-8a57-3f4bd90acb39 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:28:18.114915 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b2c7d88d-fd0c-47f1-8a57-3f4bd90acb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:28:18.174782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:28:18.230083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7d88d-fd0c-47f1-8a57-3f4bd90acb39 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:28:18.316592 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c852f56-6f1c-4ab5-948f-0b13e51bd2f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:28:18.317212 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c852f56-6f1c-4ab5-948f-0b13e51bd2f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:18.317484 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c852f56-6f1c-4ab5-948f-0b13e51bd2f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:18.317754 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c852f56-6f1c-4ab5-948f-0b13e51bd2f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:18.404333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c852f56-6f1c-4ab5-948f-0b13e51bd2f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:28:18.405769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c852f56-6f1c-4ab5-948f-0b13e51bd2f1 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:28:18.415259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 759/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:28:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:28:18.463856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1fd46-cb40-47ef-9e94-3e3b2ab86051 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:18.464366 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1fd46-cb40-47ef-9e94-3e3b2ab86051 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:18.464467 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1fd46-cb40-47ef-9e94-3e3b2ab86051 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:18.465677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 747/1506] 10.222.0.23 () {52 vars in 830 bytes} [Fri Sep 9 16:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:28:18.470706 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c852f56-6f1c-4ab5-948f-0b13e51bd2f1 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:28:18.472236 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8c852f56-6f1c-4ab5-948f-0b13e51bd2f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:28:18.492641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:28:18.593954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c852f56-6f1c-4ab5-948f-0b13e51bd2f1 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:28:18.670948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:28:18.671588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:18.671960 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:18.672229 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] PATH_INFO: `/v3/users/b49e00847418493ebddba2a76a5b3b9d` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:18.673502 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b49e00847418493ebddba2a76a5b3b9d)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:28:18.674592 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: WARNING py.warnings [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:28:18.674592 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: warnings.warn(msg) Sep 09 16:28:18.674592 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: Sep 09 16:28:18.675797 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:28:18.746855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b49e00847418493ebddba2a76a5b3b9d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:28:18.747391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b49e00847418493ebddba2a76a5b3b9d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:28:18.748109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b49e00847418493ebddba2a76a5b3b9d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:28:18.763987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b49e00847418493ebddba2a76a5b3b9d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:28:18.781572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:28:18.788901 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b49e00847418493ebddba2a76a5b3b9d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:28:18.805146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 760/1507] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:28:18 2022] DELETE /identity/v3/users/b49e00847418493ebddba2a76a5b3b9d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:28:18.809797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149e2db-ac4c-4d9f-89df-e0c49aed1149 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:18.810082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149e2db-ac4c-4d9f-89df-e0c49aed1149 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:18.810353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149e2db-ac4c-4d9f-89df-e0c49aed1149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:18.850346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a149e2db-ac4c-4d9f-89df-e0c49aed1149 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:28:18.851581 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a149e2db-ac4c-4d9f-89df-e0c49aed1149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:28:18.910760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 748/1508] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:28:18.920596 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:28:18.922207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:28:19.005301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6837c6-84c0-47cc-90b1-88fef64eeeb0 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:28:19.009549 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f35dcb-1fba-405e-9792-788b734ef79b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:28:19.010163 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f35dcb-1fba-405e-9792-788b734ef79b service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:19.010444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f35dcb-1fba-405e-9792-788b734ef79b service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:19.010747 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f35dcb-1fba-405e-9792-788b734ef79b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:19.020223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f35dcb-1fba-405e-9792-788b734ef79b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:28:19.021666 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f35dcb-1fba-405e-9792-788b734ef79b service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:28:19.031765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 761/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:28:19.389319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149e2db-ac4c-4d9f-89df-e0c49aed1149 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:28:19.391457 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a149e2db-ac4c-4d9f-89df-e0c49aed1149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:28:19.404044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149e2db-ac4c-4d9f-89df-e0c49aed1149 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:28:19.410549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d32d2-429c-4850-a508-2a4687583a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:28:19.411134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d32d2-429c-4850-a508-2a4687583a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:28:19.411550 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d32d2-429c-4850-a508-2a4687583a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:28:19.411824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d32d2-429c-4850-a508-2a4687583a79 admin admin] PATH_INFO: `/v3/projects/a59a3d120687475e947c2c97ca538c25` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:28:19.413806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d32d2-429c-4850-a508-2a4687583a79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a59a3d120687475e947c2c97ca538c25)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:28:19.414877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-437d32d2-429c-4850-a508-2a4687583a79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:28:19.414877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Sep 09 16:28:19.414877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: Sep 09 16:28:19.416117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d32d2-429c-4850-a508-2a4687583a79 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:28:19.486410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:28:19.488363 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-437d32d2-429c-4850-a508-2a4687583a79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a59a3d120687475e947c2c97ca538c25', 'request_id': 'req-437d32d2-429c-4850-a508-2a4687583a79'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:28:19.507149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-437d32d2-429c-4850-a508-2a4687583a79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a59a3d120687475e947c2c97ca538c25', 'request_id': 'req-437d32d2-429c-4850-a508-2a4687583a79'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:28:19.514597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-437d32d2-429c-4850-a508-2a4687583a79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a59a3d120687475e947c2c97ca538c25', 'request_id': 'req-437d32d2-429c-4850-a508-2a4687583a79'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:28:19.531397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-437d32d2-429c-4850-a508-2a4687583a79 admin admin] The token cache is being invalidate because project a59a3d120687475e947c2c97ca538c25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:28:19.531687 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-437d32d2-429c-4850-a508-2a4687583a79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:28:19.532983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-437d32d2-429c-4850-a508-2a4687583a79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:28:19.534835 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 749/1510] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:28:19 2022] DELETE /identity/v3/projects/a59a3d120687475e947c2c97ca538c25 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:28:29.026962 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=39,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:28:29.528215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=7,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:29:44.887826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0488a8f-bd37-4f5a-a889-bf62f05ff216 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:44.888836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0488a8f-bd37-4f5a-a889-bf62f05ff216 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:44.888836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0488a8f-bd37-4f5a-a889-bf62f05ff216 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:44.891051 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 16:29:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:29:44.897165 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38537f-d369-41de-9f5c-3b18267b2c02 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:44.897895 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38537f-d369-41de-9f5c-3b18267b2c02 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:44.898056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38537f-d369-41de-9f5c-3b18267b2c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:44.945004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-9f38537f-d369-41de-9f5c-3b18267b2c02 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:29:44.946211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9f38537f-d369-41de-9f5c-3b18267b2c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:29:45.011724 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 16:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:29:45.167754 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0488a8f-bd37-4f5a-a889-bf62f05ff216 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:29:45.171083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b0488a8f-bd37-4f5a-a889-bf62f05ff216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:29:45.287288 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0488a8f-bd37-4f5a-a889-bf62f05ff216 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:29:45.293299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8820b61-3c38-412f-91b9-3af6fcab4f7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:29:45.293906 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8820b61-3c38-412f-91b9-3af6fcab4f7b service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:45.294178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8820b61-3c38-412f-91b9-3af6fcab4f7b service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:45.294440 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8820b61-3c38-412f-91b9-3af6fcab4f7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:45.303082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8820b61-3c38-412f-91b9-3af6fcab4f7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:29:45.304658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8820b61-3c38-412f-91b9-3af6fcab4f7b service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:29:45.314082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 763/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:29:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:29:46.393837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab6f65-8da3-4695-8140-86f84f08ff90 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:46.394433 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab6f65-8da3-4695-8140-86f84f08ff90 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:46.394433 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab6f65-8da3-4695-8140-86f84f08ff90 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:46.395787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 16:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:29:46.403034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9501c8d-ecde-4dcb-b6f6-07dfc5baa820 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:46.403756 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9501c8d-ecde-4dcb-b6f6-07dfc5baa820 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:46.403946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9501c8d-ecde-4dcb-b6f6-07dfc5baa820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:46.426106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-a9501c8d-ecde-4dcb-b6f6-07dfc5baa820 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:29:46.427692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a9501c8d-ecde-4dcb-b6f6-07dfc5baa820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:29:46.436654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 16:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:29:46.595242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab6f65-8da3-4695-8140-86f84f08ff90 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:29:46.610208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab6f65-8da3-4695-8140-86f84f08ff90 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:29:46.615612 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63adb56-fd7c-434e-8f2c-085c94cd7c7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:29:46.616204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63adb56-fd7c-434e-8f2c-085c94cd7c7f service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:46.616478 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63adb56-fd7c-434e-8f2c-085c94cd7c7f service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:46.616720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63adb56-fd7c-434e-8f2c-085c94cd7c7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:46.625185 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63adb56-fd7c-434e-8f2c-085c94cd7c7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:29:46.626856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63adb56-fd7c-434e-8f2c-085c94cd7c7f service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:29:46.636294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 752/1516] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:29:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:29:47.184522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51eb685-6b03-4aff-a42b-96c5cc4b7524 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:47.184806 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51eb685-6b03-4aff-a42b-96c5cc4b7524 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:47.185082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51eb685-6b03-4aff-a42b-96c5cc4b7524 None None] PATH_INFO: `/v3` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:47.186631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 765/1517] 10.222.0.25 () {54 vars in 925 bytes} [Fri Sep 9 16:29:47 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:29:47.195847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51bdeb-6e2c-4058-9a07-f18ea0e9d285 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:47.195847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51bdeb-6e2c-4058-9a07-f18ea0e9d285 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:47.195847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51bdeb-6e2c-4058-9a07-f18ea0e9d285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:47.244814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8a51bdeb-6e2c-4058-9a07-f18ea0e9d285 None None] MFA Rules not processed for user `784b6147de91435292e4cf3940bcad9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:29:47.246020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8a51bdeb-6e2c-4058-9a07-f18ea0e9d285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:29:47.320270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 753/1518] 10.222.0.25 () {58 vars in 1015 bytes} [Fri Sep 9 16:29:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:29:50.841338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3641e-3cb0-4fa6-98ea-130b9a4625f4 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:50.841782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3641e-3cb0-4fa6-98ea-130b9a4625f4 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:50.841884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3641e-3cb0-4fa6-98ea-130b9a4625f4 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:50.843294 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 766/1519] 10.222.0.23 () {54 vars in 961 bytes} [Fri Sep 9 16:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:29:50.848410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5231cb-e7a9-416f-991d-427211248fb1 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:50.848829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5231cb-e7a9-416f-991d-427211248fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:50.848940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5231cb-e7a9-416f-991d-427211248fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:50.868848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5e5231cb-e7a9-416f-991d-427211248fb1 None None] MFA Rules not processed for user `784b6147de91435292e4cf3940bcad9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:29:50.870022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5e5231cb-e7a9-416f-991d-427211248fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:29:50.878598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 754/1520] 10.222.0.23 () {58 vars in 1058 bytes} [Fri Sep 9 16:29:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:29:50.886474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3641e-3cb0-4fa6-98ea-130b9a4625f4 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:29:50.900386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3641e-3cb0-4fa6-98ea-130b9a4625f4 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:29:50.905719 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ef57a-1be2-4404-8a34-13ecffd55ee5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784b6147de91435292e4cf3940bcad9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:29:50.906300 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ef57a-1be2-4404-8a34-13ecffd55ee5 service placement] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:50.906609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ef57a-1be2-4404-8a34-13ecffd55ee5 service placement] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:50.906882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ef57a-1be2-4404-8a34-13ecffd55ee5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:50.916084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ef57a-1be2-4404-8a34-13ecffd55ee5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:29:50.917669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ef57a-1be2-4404-8a34-13ecffd55ee5 service placement] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:29:50.927949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 767/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:29:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:29:58.137111 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041af6b2-7fa6-41f9-adb2-0201acb02b60 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:58.137585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041af6b2-7fa6-41f9-adb2-0201acb02b60 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:58.137983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041af6b2-7fa6-41f9-adb2-0201acb02b60 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:58.139400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 16:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:29:58.145929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b90099-7922-4863-a6f7-d9c325d5e15f None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:58.146416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b90099-7922-4863-a6f7-d9c325d5e15f None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:58.146416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b90099-7922-4863-a6f7-d9c325d5e15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:58.166945 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-29b90099-7922-4863-a6f7-d9c325d5e15f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:29:58.168158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-29b90099-7922-4863-a6f7-d9c325d5e15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:29:58.177938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 16:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:29:58.331597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041af6b2-7fa6-41f9-adb2-0201acb02b60 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:29:58.352727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041af6b2-7fa6-41f9-adb2-0201acb02b60 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:29:58.358944 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c688630-9562-4ea6-acdf-d0f94ca5e045 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:29:58.359543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c688630-9562-4ea6-acdf-d0f94ca5e045 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:58.359812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c688630-9562-4ea6-acdf-d0f94ca5e045 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:58.360074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c688630-9562-4ea6-acdf-d0f94ca5e045 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:58.368904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c688630-9562-4ea6-acdf-d0f94ca5e045 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:29:58.370532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c688630-9562-4ea6-acdf-d0f94ca5e045 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:29:58.380301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 756/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:29:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:29:59.453177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f6c23-d2ca-4c4e-b613-35a96ca15eb0 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:59.453791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f6c23-d2ca-4c4e-b613-35a96ca15eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:59.453791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f6c23-d2ca-4c4e-b613-35a96ca15eb0 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:59.455112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 16:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:29:59.463088 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63ad24-dbe3-41fe-8f8f-d78a727f8ed0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:59.463798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63ad24-dbe3-41fe-8f8f-d78a727f8ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:59.463968 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63ad24-dbe3-41fe-8f8f-d78a727f8ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:59.505198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3e63ad24-dbe3-41fe-8f8f-d78a727f8ed0 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:29:59.507040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3e63ad24-dbe3-41fe-8f8f-d78a727f8ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:29:59.520460 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 16:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:29:59.683997 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f6c23-d2ca-4c4e-b613-35a96ca15eb0 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:29:59.708172 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f6c23-d2ca-4c4e-b613-35a96ca15eb0 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:29:59.713755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591064a7-2023-4472-890b-595940793a9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:29:59.714357 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591064a7-2023-4472-890b-595940793a9d service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:29:59.714659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591064a7-2023-4472-890b-595940793a9d service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:29:59.714926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591064a7-2023-4472-890b-595940793a9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:29:59.723433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591064a7-2023-4472-890b-595940793a9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:29:59.725020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591064a7-2023-4472-890b-595940793a9d service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:29:59.734719 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 770/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:29:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:30:03.749089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c8f81-0d21-4325-a79f-a7bf1d851875 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:30:03.749708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c8f81-0d21-4325-a79f-a7bf1d851875 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:30:03.749708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c8f81-0d21-4325-a79f-a7bf1d851875 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:30:03.751023 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:30:03.757443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7643d06-be7d-4eff-bb3e-b98d199d77cb None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:30:03.757883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7643d06-be7d-4eff-bb3e-b98d199d77cb None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:30:03.758002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7643d06-be7d-4eff-bb3e-b98d199d77cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:30:03.788954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-c7643d06-be7d-4eff-bb3e-b98d199d77cb None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:30:03.790165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c7643d06-be7d-4eff-bb3e-b98d199d77cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:30:03.799072 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:30:04.090763 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c8f81-0d21-4325-a79f-a7bf1d851875 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:30:04.092783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f24c8f81-0d21-4325-a79f-a7bf1d851875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:30:04.105843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:30:04.153184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c8f81-0d21-4325-a79f-a7bf1d851875 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:30:04.157048 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc61cdc-b6dc-4501-ab32-85034f5ff264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:30:04.157649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc61cdc-b6dc-4501-ab32-85034f5ff264 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:30:04.157920 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc61cdc-b6dc-4501-ab32-85034f5ff264 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:30:04.158183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc61cdc-b6dc-4501-ab32-85034f5ff264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:30:04.167368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc61cdc-b6dc-4501-ab32-85034f5ff264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:30:04.168790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc61cdc-b6dc-4501-ab32-85034f5ff264 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:30:04.178559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 759/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:30:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:30:06.330724 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b00a1d-6cbc-46d7-b570-61bf1cc2eb92 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:30:06.331279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b00a1d-6cbc-46d7-b570-61bf1cc2eb92 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:30:06.331279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b00a1d-6cbc-46d7-b570-61bf1cc2eb92 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:30:06.333011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:30:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:30:06.339472 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c18ff-4be5-4ac9-94e6-1e7aee533cfd None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:30:06.339864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c18ff-4be5-4ac9-94e6-1e7aee533cfd None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:30:06.339967 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c18ff-4be5-4ac9-94e6-1e7aee533cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:30:06.367707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-830c18ff-4be5-4ac9-94e6-1e7aee533cfd None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:30:06.368876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-830c18ff-4be5-4ac9-94e6-1e7aee533cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:30:06.377064 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:30:06.673973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b00a1d-6cbc-46d7-b570-61bf1cc2eb92 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:30:06.676896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-28b00a1d-6cbc-46d7-b570-61bf1cc2eb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:30:06.696104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b00a1d-6cbc-46d7-b570-61bf1cc2eb92 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:30:06.700226 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21d036-714d-4731-a52e-3cc57b15613c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:30:06.700836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21d036-714d-4731-a52e-3cc57b15613c service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:30:06.701111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21d036-714d-4731-a52e-3cc57b15613c service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:30:06.701377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21d036-714d-4731-a52e-3cc57b15613c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:30:06.710488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21d036-714d-4731-a52e-3cc57b15613c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:30:06.711939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21d036-714d-4731-a52e-3cc57b15613c service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:30:06.721273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 773/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:30:10.457776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65947816-9147-4e26-a021-f7d4803262ec None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:30:10.458421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65947816-9147-4e26-a021-f7d4803262ec None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:30:10.458421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65947816-9147-4e26-a021-f7d4803262ec None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:30:10.459725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:30:10.466164 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a056a20-e57e-44ef-93b4-49c832fa40f5 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:30:10.467038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a056a20-e57e-44ef-93b4-49c832fa40f5 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:30:10.467038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a056a20-e57e-44ef-93b4-49c832fa40f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:30:10.478685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=74 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:30:10.491489 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-6a056a20-e57e-44ef-93b4-49c832fa40f5 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:30:10.492705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-6a056a20-e57e-44ef-93b4-49c832fa40f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:30:10.500970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:30:10.807284 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65947816-9147-4e26-a021-f7d4803262ec None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:30:10.809595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-65947816-9147-4e26-a021-f7d4803262ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:30:10.869128 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65947816-9147-4e26-a021-f7d4803262ec None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:30:10.872991 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4f43e-3802-4396-accd-bfd2dd79677a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:30:10.873594 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4f43e-3802-4396-accd-bfd2dd79677a service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:30:10.873871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4f43e-3802-4396-accd-bfd2dd79677a service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:30:10.874151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4f43e-3802-4396-accd-bfd2dd79677a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:30:10.883396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4f43e-3802-4396-accd-bfd2dd79677a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:30:10.884817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4f43e-3802-4396-accd-bfd2dd79677a service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:30:10.894979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 762/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:30:13.040287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ea02e-2bc0-4eea-9fea-bc4b6be0e6b9 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:30:13.040824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ea02e-2bc0-4eea-9fea-bc4b6be0e6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:30:13.040824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ea02e-2bc0-4eea-9fea-bc4b6be0e6b9 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:30:13.042189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 16:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:30:13.048566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63899f60-83d2-46e3-9a98-3cff91e66ea4 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:30:13.049024 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63899f60-83d2-46e3-9a98-3cff91e66ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:30:13.049129 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63899f60-83d2-46e3-9a98-3cff91e66ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:30:13.072288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-63899f60-83d2-46e3-9a98-3cff91e66ea4 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:30:13.073538 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-63899f60-83d2-46e3-9a98-3cff91e66ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:30:13.083133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 16:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:30:13.397714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ea02e-2bc0-4eea-9fea-bc4b6be0e6b9 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:30:13.399890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d18ea02e-2bc0-4eea-9fea-bc4b6be0e6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:30:13.414783 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ea02e-2bc0-4eea-9fea-bc4b6be0e6b9 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:30:13.418633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be1737-eb0b-4a84-aec2-47e6aa2919c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:30:13.419260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be1737-eb0b-4a84-aec2-47e6aa2919c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:30:13.419534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be1737-eb0b-4a84-aec2-47e6aa2919c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:30:13.419801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be1737-eb0b-4a84-aec2-47e6aa2919c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:30:13.429263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be1737-eb0b-4a84-aec2-47e6aa2919c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:30:13.430712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be1737-eb0b-4a84-aec2-47e6aa2919c6 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:30:13.439992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 776/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:30:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:30:23.066812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=54 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:30:23.436108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=8 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:35.798739 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a83dab-f1ab-4595-bbf1-df759b3b9e0a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:35.799442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a83dab-f1ab-4595-bbf1-df759b3b9e0a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:35.799442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a83dab-f1ab-4595-bbf1-df759b3b9e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:35.827942 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-36a83dab-f1ab-4595-bbf1-df759b3b9e0a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:35.829324 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-36a83dab-f1ab-4595-bbf1-df759b3b9e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:35.841154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 764/1540] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:31:35.850015 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb241c8-80d0-4d78-a55d-8dd247b32d7d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:35.850631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb241c8-80d0-4d78-a55d-8dd247b32d7d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:35.850631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb241c8-80d0-4d78-a55d-8dd247b32d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:35.879778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-3eb241c8-80d0-4d78-a55d-8dd247b32d7d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:35.881054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-3eb241c8-80d0-4d78-a55d-8dd247b32d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:35.884691 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e661cc-809b-4b06-b406-803a8ebb8070 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:35.884973 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e661cc-809b-4b06-b406-803a8ebb8070 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:35.885356 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e661cc-809b-4b06-b406-803a8ebb8070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:35.892487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 777/1541] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:35.893731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533df83-03ab-4cfe-8219-3a4994322f3a None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:35.894144 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533df83-03ab-4cfe-8219-3a4994322f3a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:35.894426 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533df83-03ab-4cfe-8219-3a4994322f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:35.916941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-96e661cc-809b-4b06-b406-803a8ebb8070 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:35.918213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-96e661cc-809b-4b06-b406-803a8ebb8070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:35.932526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-0533df83-03ab-4cfe-8219-3a4994322f3a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:35.933085 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 765/1542] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:31:35.933745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0533df83-03ab-4cfe-8219-3a4994322f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:35.933961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e661cc-809b-4b06-b406-803a8ebb8070 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:35.952045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e661cc-809b-4b06-b406-803a8ebb8070 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:35.954601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 778/1543] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:35.955899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62f081-b537-4da1-93e1-18e6246b1db2 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:35.956187 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62f081-b537-4da1-93e1-18e6246b1db2 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:35.956457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62f081-b537-4da1-93e1-18e6246b1db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:35.968588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b111f-9a9b-4aa8-b3df-179cd1fb505e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:35.969110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b111f-9a9b-4aa8-b3df-179cd1fb505e admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:35.969436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b111f-9a9b-4aa8-b3df-179cd1fb505e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:35.969712 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b111f-9a9b-4aa8-b3df-179cd1fb505e admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:35.970411 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b111f-9a9b-4aa8-b3df-179cd1fb505e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:31:35.970736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b111f-9a9b-4aa8-b3df-179cd1fb505e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:35.972411 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b111f-9a9b-4aa8-b3df-179cd1fb505e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.001493 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 766/1544] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:31:36.001911 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-1c62f081-b537-4da1-93e1-18e6246b1db2 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:36.003133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1c62f081-b537-4da1-93e1-18e6246b1db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:36.003790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22500379-ec86-434c-aa77-8d6c323ce87c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.004461 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22500379-ec86-434c-aa77-8d6c323ce87c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.004956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22500379-ec86-434c-aa77-8d6c323ce87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.023439 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 779/1545] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:36.031759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a9e48-f4ba-4118-a71b-36bc6d71236e None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.032395 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a9e48-f4ba-4118-a71b-36bc6d71236e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.032669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a9e48-f4ba-4118-a71b-36bc6d71236e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.044831 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-22500379-ec86-434c-aa77-8d6c323ce87c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:36.046053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-22500379-ec86-434c-aa77-8d6c323ce87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:36.058995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 767/1546] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:36.060533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e5537-fe1e-4fe2-86de-322ef8a5b087 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.061092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e5537-fe1e-4fe2-86de-322ef8a5b087 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.061576 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e5537-fe1e-4fe2-86de-322ef8a5b087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.062202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-028a9e48-f4ba-4118-a71b-36bc6d71236e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:36.063684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-028a9e48-f4ba-4118-a71b-36bc6d71236e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:36.072139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 780/1547] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:36.077036 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a9e48-f4ba-4118-a71b-36bc6d71236e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.088113 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e55e5537-fe1e-4fe2-86de-322ef8a5b087 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:36.089856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e55e5537-fe1e-4fe2-86de-322ef8a5b087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:36.093439 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a9e48-f4ba-4118-a71b-36bc6d71236e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.098833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c9254-6b87-4a32-81bc-0e3fdfaa8ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.099458 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c9254-6b87-4a32-81bc-0e3fdfaa8ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.099735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c9254-6b87-4a32-81bc-0e3fdfaa8ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.100002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c9254-6b87-4a32-81bc-0e3fdfaa8ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.100666 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c9254-6b87-4a32-81bc-0e3fdfaa8ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.101693 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 768/1548] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:36.102719 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c9254-6b87-4a32-81bc-0e3fdfaa8ea7 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.103184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3044ef-eda5-414e-9bb3-ab657e2511cb None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.103469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3044ef-eda5-414e-9bb3-ab657e2511cb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.103735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3044ef-eda5-414e-9bb3-ab657e2511cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.128320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8d3044ef-eda5-414e-9bb3-ab657e2511cb None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:36.129561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8d3044ef-eda5-414e-9bb3-ab657e2511cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:36.138250 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 769/1549] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:36.139502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de49b4d-b06d-4d7c-904d-16c505f7ef21 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.139762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de49b4d-b06d-4d7c-904d-16c505f7ef21 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.140037 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de49b4d-b06d-4d7c-904d-16c505f7ef21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.145460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 781/1550] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:36.145460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c9254-6b87-4a32-81bc-0e3fdfaa8ea7 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.159243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3de49b4d-b06d-4d7c-904d-16c505f7ef21 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:36.160459 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3de49b4d-b06d-4d7c-904d-16c505f7ef21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:36.205040 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c9254-6b87-4a32-81bc-0e3fdfaa8ea7 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.210806 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391a84e-4d1d-4403-ad18-5b387b055e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.211484 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391a84e-4d1d-4403-ad18-5b387b055e5b admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.211761 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391a84e-4d1d-4403-ad18-5b387b055e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.212032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391a84e-4d1d-4403-ad18-5b387b055e5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.212678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391a84e-4d1d-4403-ad18-5b387b055e5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:31:36.212976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391a84e-4d1d-4403-ad18-5b387b055e5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.214002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: WARNING py.warnings [None req-8391a84e-4d1d-4403-ad18-5b387b055e5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:31:36.214002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: warnings.warn(msg) Sep 09 16:31:36.214002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: Sep 09 16:31:36.215067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391a84e-4d1d-4403-ad18-5b387b055e5b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.218491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 770/1551] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:36.219365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de49b4d-b06d-4d7c-904d-16c505f7ef21 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.227965 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 782/1552] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:31:36.229200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80131359-3408-4288-82ce-c7fc3284391e None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.229351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de49b4d-b06d-4d7c-904d-16c505f7ef21 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.229507 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80131359-3408-4288-82ce-c7fc3284391e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.229752 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80131359-3408-4288-82ce-c7fc3284391e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.238443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea84842-1d80-45dc-92c7-36d8e9dfadf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.239152 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea84842-1d80-45dc-92c7-36d8e9dfadf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.239426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea84842-1d80-45dc-92c7-36d8e9dfadf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.239694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea84842-1d80-45dc-92c7-36d8e9dfadf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.240339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea84842-1d80-45dc-92c7-36d8e9dfadf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:31:36.240683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea84842-1d80-45dc-92c7-36d8e9dfadf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.242275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea84842-1d80-45dc-92c7-36d8e9dfadf6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.251998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-80131359-3408-4288-82ce-c7fc3284391e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:36.253199 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-80131359-3408-4288-82ce-c7fc3284391e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:36.253900 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 771/1553] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:31:36.255173 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2d4fc-a32c-497d-86b7-c9b309cf2385 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.255459 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2d4fc-a32c-497d-86b7-c9b309cf2385 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.255733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2d4fc-a32c-497d-86b7-c9b309cf2385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.261992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 783/1554] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:36.263423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c37b1-af9a-419a-93cc-bbf91e096b34 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.263701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c37b1-af9a-419a-93cc-bbf91e096b34 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.263971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c37b1-af9a-419a-93cc-bbf91e096b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.280151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8ab2d4fc-a32c-497d-86b7-c9b309cf2385 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:36.281317 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8ab2d4fc-a32c-497d-86b7-c9b309cf2385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:36.283374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f85c37b1-af9a-419a-93cc-bbf91e096b34 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:36.284515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f85c37b1-af9a-419a-93cc-bbf91e096b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:36.289305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 772/1555] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:36.290079 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2d4fc-a32c-497d-86b7-c9b309cf2385 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.299407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2d4fc-a32c-497d-86b7-c9b309cf2385 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.299585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 784/1556] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:36.300831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c37b1-af9a-419a-93cc-bbf91e096b34 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.307043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4931b9f-080e-4734-88f3-24565ad2b553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.307885 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4931b9f-080e-4734-88f3-24565ad2b553 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.308400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4931b9f-080e-4734-88f3-24565ad2b553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.308924 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4931b9f-080e-4734-88f3-24565ad2b553 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.309941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4931b9f-080e-4734-88f3-24565ad2b553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:31:36.310188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4931b9f-080e-4734-88f3-24565ad2b553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.310948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c37b1-af9a-419a-93cc-bbf91e096b34 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.312105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4931b9f-080e-4734-88f3-24565ad2b553 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.316050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa88fca-f7c5-49ef-b7fd-27fae1528ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.316647 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa88fca-f7c5-49ef-b7fd-27fae1528ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.316958 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa88fca-f7c5-49ef-b7fd-27fae1528ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.317243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa88fca-f7c5-49ef-b7fd-27fae1528ec6 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.317976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa88fca-f7c5-49ef-b7fd-27fae1528ec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.319494 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: WARNING py.warnings [None req-0fa88fca-f7c5-49ef-b7fd-27fae1528ec6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:31:36.319494 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: warnings.warn(msg) Sep 09 16:31:36.319494 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: Sep 09 16:31:36.320731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa88fca-f7c5-49ef-b7fd-27fae1528ec6 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.326189 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 773/1557] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:31:36.327454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5feda-476e-4b3e-9606-727184f9dd7d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.327734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5feda-476e-4b3e-9606-727184f9dd7d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.327998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5feda-476e-4b3e-9606-727184f9dd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.348007 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-00f5feda-476e-4b3e-9606-727184f9dd7d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:36.349223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-00f5feda-476e-4b3e-9606-727184f9dd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:36.355178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 785/1558] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/users => generated 289 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:36.355178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa88fca-f7c5-49ef-b7fd-27fae1528ec6 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.360658 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 774/1559] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:36.362938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa88fca-f7c5-49ef-b7fd-27fae1528ec6 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.363976 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5feda-476e-4b3e-9606-727184f9dd7d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.367729 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6cffd-cb73-4804-bf27-7defddbdfd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.368326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6cffd-cb73-4804-bf27-7defddbdfd2d admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.368604 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6cffd-cb73-4804-bf27-7defddbdfd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.368869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6cffd-cb73-4804-bf27-7defddbdfd2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.369487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6cffd-cb73-4804-bf27-7defddbdfd2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.371419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6cffd-cb73-4804-bf27-7defddbdfd2d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.373741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5feda-476e-4b3e-9606-727184f9dd7d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.378288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c95b3-7de4-4853-b2fc-8542dbcee1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.378910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c95b3-7de4-4853-b2fc-8542dbcee1fa admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.379186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c95b3-7de4-4853-b2fc-8542dbcee1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.379486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c95b3-7de4-4853-b2fc-8542dbcee1fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.380098 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c95b3-7de4-4853-b2fc-8542dbcee1fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.381102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-952c95b3-7de4-4853-b2fc-8542dbcee1fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:31:36.381102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Sep 09 16:31:36.381102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: Sep 09 16:31:36.382328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c95b3-7de4-4853-b2fc-8542dbcee1fa admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.399914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 786/1560] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:36.401333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d69fb-36d5-4c61-867c-385d31014449 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.401612 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d69fb-36d5-4c61-867c-385d31014449 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.401878 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d69fb-36d5-4c61-867c-385d31014449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.412961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 775/1561] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/projects => generated 401 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:36.414242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26ef43-5410-47e2-b66d-57dd6f294239 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.414550 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26ef43-5410-47e2-b66d-57dd6f294239 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.414828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26ef43-5410-47e2-b66d-57dd6f294239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.422074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-444d69fb-36d5-4c61-867c-385d31014449 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:36.423346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-444d69fb-36d5-4c61-867c-385d31014449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:36.435662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5f26ef43-5410-47e2-b66d-57dd6f294239 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:36.436838 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5f26ef43-5410-47e2-b66d-57dd6f294239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:36.463695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:36.484570 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:36.486189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 787/1562] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:36.487223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d69fb-36d5-4c61-867c-385d31014449 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.499616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 776/1563] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:36.501013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26ef43-5410-47e2-b66d-57dd6f294239 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.509729 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d69fb-36d5-4c61-867c-385d31014449 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.514107 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26ef43-5410-47e2-b66d-57dd6f294239 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.515067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186eb94c-f1a8-4a02-be9a-9a9a769eb420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.515700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb94c-f1a8-4a02-be9a-9a9a769eb420 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.515979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb94c-f1a8-4a02-be9a-9a9a769eb420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.516250 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb94c-f1a8-4a02-be9a-9a9a769eb420 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.516926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186eb94c-f1a8-4a02-be9a-9a9a769eb420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:31:36.517230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186eb94c-f1a8-4a02-be9a-9a9a769eb420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.518875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186eb94c-f1a8-4a02-be9a-9a9a769eb420 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.519018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67f8cd-d587-4ade-82e0-dd8883869109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.519502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67f8cd-d587-4ade-82e0-dd8883869109 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.519779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67f8cd-d587-4ade-82e0-dd8883869109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.520074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67f8cd-d587-4ade-82e0-dd8883869109 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.520706 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67f8cd-d587-4ade-82e0-dd8883869109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.522695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67f8cd-d587-4ade-82e0-dd8883869109 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.531259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 788/1564] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:31:36.532425 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186eb94c-f1a8-4a02-be9a-9a9a769eb420 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.542224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186eb94c-f1a8-4a02-be9a-9a9a769eb420 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.547357 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8bb82-a61e-44f5-8372-7df8bfc1e07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.547966 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8bb82-a61e-44f5-8372-7df8bfc1e07b admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.548250 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8bb82-a61e-44f5-8372-7df8bfc1e07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.548519 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8bb82-a61e-44f5-8372-7df8bfc1e07b admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.549160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8bb82-a61e-44f5-8372-7df8bfc1e07b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.551189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8bb82-a61e-44f5-8372-7df8bfc1e07b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.555673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 777/1565] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:36.555673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67f8cd-d587-4ade-82e0-dd8883869109 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.565133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 789/1566] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:36.565133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8bb82-a61e-44f5-8372-7df8bfc1e07b admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.574543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8bb82-a61e-44f5-8372-7df8bfc1e07b admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.621779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67f8cd-d587-4ade-82e0-dd8883869109 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.626652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8f74e-5064-4259-82b2-19415a73b410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.627263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f74e-5064-4259-82b2-19415a73b410 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.627536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f74e-5064-4259-82b2-19415a73b410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.627803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f74e-5064-4259-82b2-19415a73b410 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.628451 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8f74e-5064-4259-82b2-19415a73b410 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:31:36.628732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8f74e-5064-4259-82b2-19415a73b410 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.629389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e085767-e9b5-4643-a60f-e33e37e770af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.630079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e085767-e9b5-4643-a60f-e33e37e770af admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.630349 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e085767-e9b5-4643-a60f-e33e37e770af admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.630595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8f74e-5064-4259-82b2-19415a73b410 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.630716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e085767-e9b5-4643-a60f-e33e37e770af admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.631272 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e085767-e9b5-4643-a60f-e33e37e770af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.633210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e085767-e9b5-4643-a60f-e33e37e770af admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.642238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 778/1567] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:31:36.643130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8f74e-5064-4259-82b2-19415a73b410 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.653265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8f74e-5064-4259-82b2-19415a73b410 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.658253 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641afea-0dbc-462e-99d3-8dd46f60a7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.658874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641afea-0dbc-462e-99d3-8dd46f60a7f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.659155 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641afea-0dbc-462e-99d3-8dd46f60a7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.659416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641afea-0dbc-462e-99d3-8dd46f60a7f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.663223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 790/1568] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:36.663384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641afea-0dbc-462e-99d3-8dd46f60a7f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.663481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e085767-e9b5-4643-a60f-e33e37e770af admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.663578 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641afea-0dbc-462e-99d3-8dd46f60a7f3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.672844 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e085767-e9b5-4643-a60f-e33e37e770af admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.679895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccd335-1733-4a3a-be7e-1ad1237ffc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.680516 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccd335-1733-4a3a-be7e-1ad1237ffc62 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.680801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccd335-1733-4a3a-be7e-1ad1237ffc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.681056 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccd335-1733-4a3a-be7e-1ad1237ffc62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.681701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ccd335-1733-4a3a-be7e-1ad1237ffc62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:31:36.682001 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ccd335-1733-4a3a-be7e-1ad1237ffc62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.683813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ccd335-1733-4a3a-be7e-1ad1237ffc62 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.687250 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 779/1569] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/users => generated 301 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:36.687250 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641afea-0dbc-462e-99d3-8dd46f60a7f3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.694844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641afea-0dbc-462e-99d3-8dd46f60a7f3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.704607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 791/1570] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:31:36.704607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccd335-1733-4a3a-be7e-1ad1237ffc62 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.704880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559db4e8-9f07-4dc6-8d3b-684d29e1eaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.704880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559db4e8-9f07-4dc6-8d3b-684d29e1eaef admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.704880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559db4e8-9f07-4dc6-8d3b-684d29e1eaef admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.704880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559db4e8-9f07-4dc6-8d3b-684d29e1eaef admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.705325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559db4e8-9f07-4dc6-8d3b-684d29e1eaef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:31:36.705325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559db4e8-9f07-4dc6-8d3b-684d29e1eaef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.705325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559db4e8-9f07-4dc6-8d3b-684d29e1eaef admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.707759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccd335-1733-4a3a-be7e-1ad1237ffc62 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.712633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ec79a-0997-4cf1-aa02-6211a57d9a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.713218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ec79a-0997-4cf1-aa02-6211a57d9a1f admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.713493 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ec79a-0997-4cf1-aa02-6211a57d9a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.713752 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ec79a-0997-4cf1-aa02-6211a57d9a1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.714357 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ec79a-0997-4cf1-aa02-6211a57d9a1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.716563 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ec79a-0997-4cf1-aa02-6211a57d9a1f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.717109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 780/1571] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:31:36.717864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559db4e8-9f07-4dc6-8d3b-684d29e1eaef admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.729126 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559db4e8-9f07-4dc6-8d3b-684d29e1eaef admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.734034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95cdc5-62c8-4da0-8fbb-45d01f8f6b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.734640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95cdc5-62c8-4da0-8fbb-45d01f8f6b3c admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.734932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95cdc5-62c8-4da0-8fbb-45d01f8f6b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.735183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95cdc5-62c8-4da0-8fbb-45d01f8f6b3c admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.735785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95cdc5-62c8-4da0-8fbb-45d01f8f6b3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.737674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95cdc5-62c8-4da0-8fbb-45d01f8f6b3c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.749221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 792/1572] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:36.750040 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ec79a-0997-4cf1-aa02-6211a57d9a1f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.762356 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 781/1573] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/users => generated 286 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:36.762356 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95cdc5-62c8-4da0-8fbb-45d01f8f6b3c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.770909 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95cdc5-62c8-4da0-8fbb-45d01f8f6b3c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.814487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ec79a-0997-4cf1-aa02-6211a57d9a1f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.819801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f42ee6-e54f-419a-95ba-81ea0e80ed79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.820350 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f42ee6-e54f-419a-95ba-81ea0e80ed79 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.820716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f42ee6-e54f-419a-95ba-81ea0e80ed79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.820936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f42ee6-e54f-419a-95ba-81ea0e80ed79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.821625 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f42ee6-e54f-419a-95ba-81ea0e80ed79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.823582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f42ee6-e54f-419a-95ba-81ea0e80ed79 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.829954 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470d560-a213-4dc9-80f9-fbc3023e28f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.830727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470d560-a213-4dc9-80f9-fbc3023e28f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.831003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470d560-a213-4dc9-80f9-fbc3023e28f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.831275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470d560-a213-4dc9-80f9-fbc3023e28f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.831946 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470d560-a213-4dc9-80f9-fbc3023e28f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.833410 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 793/1574] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:36.833961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470d560-a213-4dc9-80f9-fbc3023e28f8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.834205 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f42ee6-e54f-419a-95ba-81ea0e80ed79 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.847368 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f42ee6-e54f-419a-95ba-81ea0e80ed79 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.852032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29c5eb-d280-406d-8ca9-b88964d24177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.852739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29c5eb-d280-406d-8ca9-b88964d24177 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.852892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29c5eb-d280-406d-8ca9-b88964d24177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.853138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29c5eb-d280-406d-8ca9-b88964d24177 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.853767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29c5eb-d280-406d-8ca9-b88964d24177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.855415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29c5eb-d280-406d-8ca9-b88964d24177 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.860567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:36.864459 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 782/1575] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:36.864459 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470d560-a213-4dc9-80f9-fbc3023e28f8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.867346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 794/1576] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:36.868140 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29c5eb-d280-406d-8ca9-b88964d24177 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.868606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:36.878657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29c5eb-d280-406d-8ca9-b88964d24177 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.928956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470d560-a213-4dc9-80f9-fbc3023e28f8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.933630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a622b7-273c-4956-8174-d5e0fe2d9241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.933801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04613f-364f-49f4-aa55-79575f17d4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.934295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a622b7-273c-4956-8174-d5e0fe2d9241 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.934404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04613f-364f-49f4-aa55-79575f17d4cd admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.934601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a622b7-273c-4956-8174-d5e0fe2d9241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.934701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04613f-364f-49f4-aa55-79575f17d4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.934870 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a622b7-273c-4956-8174-d5e0fe2d9241 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.934967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04613f-364f-49f4-aa55-79575f17d4cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.935508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a622b7-273c-4956-8174-d5e0fe2d9241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.935612 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04613f-364f-49f4-aa55-79575f17d4cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.937148 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a622b7-273c-4956-8174-d5e0fe2d9241 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.937394 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04613f-364f-49f4-aa55-79575f17d4cd admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.946833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 783/1577] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:36.947603 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a622b7-273c-4956-8174-d5e0fe2d9241 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.956722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a622b7-273c-4956-8174-d5e0fe2d9241 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:36.961602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1cfa20-8464-4787-ba2a-3159968cf17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:36.962191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cfa20-8464-4787-ba2a-3159968cf17a admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:36.962465 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cfa20-8464-4787-ba2a-3159968cf17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:36.962761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cfa20-8464-4787-ba2a-3159968cf17a admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:36.963371 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1cfa20-8464-4787-ba2a-3159968cf17a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:36.968031 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 795/1578] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:36.968181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1cfa20-8464-4787-ba2a-3159968cf17a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:36.968316 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04613f-364f-49f4-aa55-79575f17d4cd admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:36.997773 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 784/1579] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:36.997773 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1cfa20-8464-4787-ba2a-3159968cf17a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.038289 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04613f-364f-49f4-aa55-79575f17d4cd admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.056712 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1cfa20-8464-4787-ba2a-3159968cf17a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.062882 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7e653-60b3-4938-bf73-68fca0308516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.062882 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7e653-60b3-4938-bf73-68fca0308516 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.062882 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7e653-60b3-4938-bf73-68fca0308516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.062882 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7e653-60b3-4938-bf73-68fca0308516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.063223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7e653-60b3-4938-bf73-68fca0308516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.064659 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7e653-60b3-4938-bf73-68fca0308516 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.076296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 785/1580] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:37.076296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7e653-60b3-4938-bf73-68fca0308516 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.086865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7e653-60b3-4938-bf73-68fca0308516 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.091642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f56476-d79e-4337-a399-d160d50d5c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.092292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f56476-d79e-4337-a399-d160d50d5c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.092567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f56476-d79e-4337-a399-d160d50d5c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.092833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f56476-d79e-4337-a399-d160d50d5c9c admin admin] PATH_INFO: `/v3/projects/f90f2e0c002343d88ae25dc7bfb2fe10/users/d095ed814b9e47e08d6b5a4fd2901c43/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.098142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e25e2-cc9a-4f53-b288-afa27df223ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.098810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e25e2-cc9a-4f53-b288-afa27df223ac admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.099117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e25e2-cc9a-4f53-b288-afa27df223ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.099406 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e25e2-cc9a-4f53-b288-afa27df223ac admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.100131 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e25e2-cc9a-4f53-b288-afa27df223ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.102090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e25e2-cc9a-4f53-b288-afa27df223ac admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.108130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f56476-d79e-4337-a399-d160d50d5c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90f2e0c002343d88ae25dc7bfb2fe10, user_id=d095ed814b9e47e08d6b5a4fd2901c43, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.109186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-62f56476-d79e-4337-a399-d160d50d5c9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:31:37.109186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Sep 09 16:31:37.109186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: Sep 09 16:31:37.111151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f56476-d79e-4337-a399-d160d50d5c9c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.125262 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 796/1581] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:36 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:37.126056 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e25e2-cc9a-4f53-b288-afa27df223ac admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.126311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 786/1582] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:37 2022] PUT /identity/v3/projects/f90f2e0c002343d88ae25dc7bfb2fe10/users/d095ed814b9e47e08d6b5a4fd2901c43/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:37.127138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f56476-d79e-4337-a399-d160d50d5c9c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.141081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e25e2-cc9a-4f53-b288-afa27df223ac admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.160168 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:37.179625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:37.194880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f56476-d79e-4337-a399-d160d50d5c9c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.198886 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f46a6-6b4a-448b-980a-03d479c9129d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.199507 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f46a6-6b4a-448b-980a-03d479c9129d admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.199699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46121fc-80fc-4e8e-a5e1-433fdc81d044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.199824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f46a6-6b4a-448b-980a-03d479c9129d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.200037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f46a6-6b4a-448b-980a-03d479c9129d admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.200288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46121fc-80fc-4e8e-a5e1-433fdc81d044 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.200564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46121fc-80fc-4e8e-a5e1-433fdc81d044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.200661 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f46a6-6b4a-448b-980a-03d479c9129d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.200834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46121fc-80fc-4e8e-a5e1-433fdc81d044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.201504 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46121fc-80fc-4e8e-a5e1-433fdc81d044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.202607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f46a6-6b4a-448b-980a-03d479c9129d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.203132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46121fc-80fc-4e8e-a5e1-433fdc81d044 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.213140 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 787/1583] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:37.213950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46121fc-80fc-4e8e-a5e1-433fdc81d044 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.224453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46121fc-80fc-4e8e-a5e1-433fdc81d044 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.226917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 797/1584] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:37 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:37.227694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f46a6-6b4a-448b-980a-03d479c9129d admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.229550 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44330522-46f8-47d3-a475-86640c409b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.230247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44330522-46f8-47d3-a475-86640c409b99 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.230446 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44330522-46f8-47d3-a475-86640c409b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.230747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44330522-46f8-47d3-a475-86640c409b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.231383 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44330522-46f8-47d3-a475-86640c409b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.232979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44330522-46f8-47d3-a475-86640c409b99 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.237263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f46a6-6b4a-448b-980a-03d479c9129d admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.242212 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eee422-df0c-4ce3-9054-b61347eb4c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.242850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eee422-df0c-4ce3-9054-b61347eb4c32 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.243130 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eee422-df0c-4ce3-9054-b61347eb4c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.243279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 788/1585] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:37.243446 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eee422-df0c-4ce3-9054-b61347eb4c32 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.244040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44330522-46f8-47d3-a475-86640c409b99 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.244139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eee422-df0c-4ce3-9054-b61347eb4c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.245982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eee422-df0c-4ce3-9054-b61347eb4c32 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.253305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44330522-46f8-47d3-a475-86640c409b99 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.258941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66174314-ebfa-4aee-8b8f-80aca7463da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.259646 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66174314-ebfa-4aee-8b8f-80aca7463da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.260091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66174314-ebfa-4aee-8b8f-80aca7463da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.260430 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66174314-ebfa-4aee-8b8f-80aca7463da8 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.261236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66174314-ebfa-4aee-8b8f-80aca7463da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.263618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66174314-ebfa-4aee-8b8f-80aca7463da8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.269076 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 798/1586] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:37 2022] POST /identity/v3/users => generated 292 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:37.269910 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eee422-df0c-4ce3-9054-b61347eb4c32 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.281848 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eee422-df0c-4ce3-9054-b61347eb4c32 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.286558 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 789/1587] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:37 2022] POST /identity/v3/users => generated 290 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:37.287772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed688a-88be-4f32-88f4-dc601169eadb None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.288060 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed688a-88be-4f32-88f4-dc601169eadb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.288323 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed688a-88be-4f32-88f4-dc601169eadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.289677 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91c0e8-2486-4f0e-baa0-53a21b4f5f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.290294 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91c0e8-2486-4f0e-baa0-53a21b4f5f62 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.290601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91c0e8-2486-4f0e-baa0-53a21b4f5f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.290873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91c0e8-2486-4f0e-baa0-53a21b4f5f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.291522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91c0e8-2486-4f0e-baa0-53a21b4f5f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.293106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91c0e8-2486-4f0e-baa0-53a21b4f5f62 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.303686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 799/1588] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:37.304338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91c0e8-2486-4f0e-baa0-53a21b4f5f62 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.307881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-dbed688a-88be-4f32-88f4-dc601169eadb None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:37.309067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dbed688a-88be-4f32-88f4-dc601169eadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:37.315618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91c0e8-2486-4f0e-baa0-53a21b4f5f62 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.319946 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 790/1589] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:37.319946 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed688a-88be-4f32-88f4-dc601169eadb None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.321091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcbc4c2-6aad-407b-9014-86f50f5ee151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.321216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbc4c2-6aad-407b-9014-86f50f5ee151 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.321485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbc4c2-6aad-407b-9014-86f50f5ee151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.321755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbc4c2-6aad-407b-9014-86f50f5ee151 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.322447 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbc4c2-6aad-407b-9014-86f50f5ee151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.324000 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbc4c2-6aad-407b-9014-86f50f5ee151 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.327035 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed688a-88be-4f32-88f4-dc601169eadb None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.331597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454783c-4f8a-4bc0-98af-8b386ec564c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.332251 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454783c-4f8a-4bc0-98af-8b386ec564c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.332586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454783c-4f8a-4bc0-98af-8b386ec564c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.332857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454783c-4f8a-4bc0-98af-8b386ec564c3 admin admin] PATH_INFO: `/v3/projects/cf44e7961d7e4ffcb7289c090e551583/users/b253af9b45d146c2bd87d64576fce8dc/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.334302 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 800/1590] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:37.335131 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcbc4c2-6aad-407b-9014-86f50f5ee151 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.345889 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcbc4c2-6aad-407b-9014-86f50f5ee151 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.349083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454783c-4f8a-4bc0-98af-8b386ec564c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf44e7961d7e4ffcb7289c090e551583, user_id=b253af9b45d146c2bd87d64576fce8dc, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.350436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20d701-4227-4537-a2fa-0a990f805752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.351050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20d701-4227-4537-a2fa-0a990f805752 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.351330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20d701-4227-4537-a2fa-0a990f805752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.351601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20d701-4227-4537-a2fa-0a990f805752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.351903 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454783c-4f8a-4bc0-98af-8b386ec564c3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.352230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20d701-4227-4537-a2fa-0a990f805752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.353786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20d701-4227-4537-a2fa-0a990f805752 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.364725 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 801/1591] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:37.365577 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 791/1592] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:37 2022] PUT /identity/v3/projects/cf44e7961d7e4ffcb7289c090e551583/users/b253af9b45d146c2bd87d64576fce8dc/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:37.365708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20d701-4227-4537-a2fa-0a990f805752 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.367318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454783c-4f8a-4bc0-98af-8b386ec564c3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.377245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20d701-4227-4537-a2fa-0a990f805752 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.429829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454783c-4f8a-4bc0-98af-8b386ec564c3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.432000 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ce530-f43e-4beb-bea0-d2ad0890db21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.432680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ce530-f43e-4beb-bea0-d2ad0890db21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.432987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ce530-f43e-4beb-bea0-d2ad0890db21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.433223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ce530-f43e-4beb-bea0-d2ad0890db21 admin admin] PATH_INFO: `/v3/projects/c2965cd2c2464e79b878aa09cb1d293d/users/00f532e0d03048cf86f9406d53e71d16/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.434731 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1676985-1f35-4416-a7d0-8d1e50106094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.435508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1676985-1f35-4416-a7d0-8d1e50106094 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.435785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1676985-1f35-4416-a7d0-8d1e50106094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.436057 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1676985-1f35-4416-a7d0-8d1e50106094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.436700 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1676985-1f35-4416-a7d0-8d1e50106094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.438301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1676985-1f35-4416-a7d0-8d1e50106094 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.447088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ce530-f43e-4beb-bea0-d2ad0890db21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2965cd2c2464e79b878aa09cb1d293d, user_id=00f532e0d03048cf86f9406d53e71d16, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.448278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 792/1593] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:37.449067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1676985-1f35-4416-a7d0-8d1e50106094 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.449893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ce530-f43e-4beb-bea0-d2ad0890db21 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.458694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1676985-1f35-4416-a7d0-8d1e50106094 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.463934 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 802/1594] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:37 2022] PUT /identity/v3/projects/c2965cd2c2464e79b878aa09cb1d293d/users/00f532e0d03048cf86f9406d53e71d16/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:37.468387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ce530-f43e-4beb-bea0-d2ad0890db21 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.514486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:37.516516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86960e6d-22d2-41be-bfba-b65cf43b79c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.517410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86960e6d-22d2-41be-bfba-b65cf43b79c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.518084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86960e6d-22d2-41be-bfba-b65cf43b79c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.518084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86960e6d-22d2-41be-bfba-b65cf43b79c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.518606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86960e6d-22d2-41be-bfba-b65cf43b79c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.520512 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86960e6d-22d2-41be-bfba-b65cf43b79c3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.531161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 793/1595] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:37.532091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86960e6d-22d2-41be-bfba-b65cf43b79c3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.533130 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ce530-f43e-4beb-bea0-d2ad0890db21 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.538453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3631762-0c6c-44a9-853a-256d9b09fe56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.539137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3631762-0c6c-44a9-853a-256d9b09fe56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.539417 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3631762-0c6c-44a9-853a-256d9b09fe56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.539688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3631762-0c6c-44a9-853a-256d9b09fe56 admin admin] PATH_INFO: `/v3/projects/f187443561a4466786dc67c3b5b1b6da/users/252f2312bbfb4dcfba616f7051f6a124/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.554022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:37.555522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3631762-0c6c-44a9-853a-256d9b09fe56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f187443561a4466786dc67c3b5b1b6da, user_id=252f2312bbfb4dcfba616f7051f6a124, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.559450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3631762-0c6c-44a9-853a-256d9b09fe56 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.573506 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 803/1596] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:37 2022] PUT /identity/v3/projects/f187443561a4466786dc67c3b5b1b6da/users/252f2312bbfb4dcfba616f7051f6a124/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:37.579758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3631762-0c6c-44a9-853a-256d9b09fe56 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.591796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86960e6d-22d2-41be-bfba-b65cf43b79c3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.641124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53821212-b77a-419f-a9a4-ae86ff96c6ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.641739 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53821212-b77a-419f-a9a4-ae86ff96c6ec service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.642136 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53821212-b77a-419f-a9a4-ae86ff96c6ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.642390 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3631762-0c6c-44a9-853a-256d9b09fe56 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.642613 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53821212-b77a-419f-a9a4-ae86ff96c6ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.648009 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd575dd2-d278-4f98-bce8-320b9b5c7873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.648619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd575dd2-d278-4f98-bce8-320b9b5c7873 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.649004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd575dd2-d278-4f98-bce8-320b9b5c7873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.649276 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd575dd2-d278-4f98-bce8-320b9b5c7873 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.649925 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd575dd2-d278-4f98-bce8-320b9b5c7873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.651577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd575dd2-d278-4f98-bce8-320b9b5c7873 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.652045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53821212-b77a-419f-a9a4-ae86ff96c6ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.653479 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53821212-b77a-419f-a9a4-ae86ff96c6ec service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.661621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 804/1597] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:37.662417 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd575dd2-d278-4f98-bce8-320b9b5c7873 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.662997 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 794/1598] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:37.664433 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e13549-cfe6-4f17-9acb-c69c83f6db5c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.664941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e13549-cfe6-4f17-9acb-c69c83f6db5c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.665201 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e13549-cfe6-4f17-9acb-c69c83f6db5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.672404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd575dd2-d278-4f98-bce8-320b9b5c7873 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.677460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d16d49-c245-492e-93f8-29bf97fbdcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.678072 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d16d49-c245-492e-93f8-29bf97fbdcc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.678348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d16d49-c245-492e-93f8-29bf97fbdcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.679412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d16d49-c245-492e-93f8-29bf97fbdcc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.679412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d16d49-c245-492e-93f8-29bf97fbdcc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.681199 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d16d49-c245-492e-93f8-29bf97fbdcc4 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.693511 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 805/1599] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:37.694663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d16d49-c245-492e-93f8-29bf97fbdcc4 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.697849 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-55e13549-cfe6-4f17-9acb-c69c83f6db5c None None] MFA Rules not processed for user `b253af9b45d146c2bd87d64576fce8dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:37.699102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-55e13549-cfe6-4f17-9acb-c69c83f6db5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:37.708768 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d16d49-c245-492e-93f8-29bf97fbdcc4 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.713644 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d228a-e44a-48b0-8d0c-e294d9d2566c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.714631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d228a-e44a-48b0-8d0c-e294d9d2566c admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.715059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d228a-e44a-48b0-8d0c-e294d9d2566c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.715336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d228a-e44a-48b0-8d0c-e294d9d2566c admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.715979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d228a-e44a-48b0-8d0c-e294d9d2566c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.717946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d228a-e44a-48b0-8d0c-e294d9d2566c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.729909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 806/1600] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:37.733457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5869f-396b-4715-837f-00a0d6150774 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.733457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5869f-396b-4715-837f-00a0d6150774 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.733457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5869f-396b-4715-837f-00a0d6150774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.755377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f7c5869f-396b-4715-837f-00a0d6150774 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:37.757122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f7c5869f-396b-4715-837f-00a0d6150774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:37.765895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 807/1601] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:37.766584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5869f-396b-4715-837f-00a0d6150774 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.780620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5869f-396b-4715-837f-00a0d6150774 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.785963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430fc64-2f8e-4cd1-9769-44d5ef223cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.786625 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430fc64-2f8e-4cd1-9769-44d5ef223cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.786904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430fc64-2f8e-4cd1-9769-44d5ef223cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.787189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430fc64-2f8e-4cd1-9769-44d5ef223cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.787936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430fc64-2f8e-4cd1-9769-44d5ef223cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.789578 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430fc64-2f8e-4cd1-9769-44d5ef223cb9 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.794662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 795/1602] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:37.794662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ece35-5024-4c37-ac89-2f45c66ae030 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.794662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ece35-5024-4c37-ac89-2f45c66ae030 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.794662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ece35-5024-4c37-ac89-2f45c66ae030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.804605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 808/1603] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:37.805363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430fc64-2f8e-4cd1-9769-44d5ef223cb9 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.815496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430fc64-2f8e-4cd1-9769-44d5ef223cb9 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.820384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62317d-c5f1-4553-9403-758508d39151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.821056 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62317d-c5f1-4553-9403-758508d39151 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.821325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62317d-c5f1-4553-9403-758508d39151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.821591 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62317d-c5f1-4553-9403-758508d39151 admin admin] PATH_INFO: `/v3/projects/75e4208363cf47dfb52c9a5313068435/users/a89acf33520f4842ad411207309ebf9e/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.826966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-9d0ece35-5024-4c37-ac89-2f45c66ae030 None None] MFA Rules not processed for user `252f2312bbfb4dcfba616f7051f6a124`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:37.828345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9d0ece35-5024-4c37-ac89-2f45c66ae030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:37.838642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62317d-c5f1-4553-9403-758508d39151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75e4208363cf47dfb52c9a5313068435, user_id=a89acf33520f4842ad411207309ebf9e, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.841436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62317d-c5f1-4553-9403-758508d39151 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:37.861546 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 809/1604] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:37 2022] PUT /identity/v3/projects/75e4208363cf47dfb52c9a5313068435/users/a89acf33520f4842ad411207309ebf9e/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:37.863328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62317d-c5f1-4553-9403-758508d39151 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.901739 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:37.919017 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 796/1605] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:37.920413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ece35-5024-4c37-ac89-2f45c66ae030 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:37.932557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ece35-5024-4c37-ac89-2f45c66ae030 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.958946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62317d-c5f1-4553-9403-758508d39151 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:37.964533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c0edf-9a22-4b83-9aee-2340971d1df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:37.965365 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c0edf-9a22-4b83-9aee-2340971d1df5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:37.965647 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c0edf-9a22-4b83-9aee-2340971d1df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:37.965918 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c0edf-9a22-4b83-9aee-2340971d1df5 admin admin] PATH_INFO: `/v3/projects/d21ab544d0b547919794e3efb49ba0e9/users/67d86f09c6304121be2021cb5c15e63e/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:37.969854 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:37.985275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c0edf-9a22-4b83-9aee-2340971d1df5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d21ab544d0b547919794e3efb49ba0e9, user_id=67d86f09c6304121be2021cb5c15e63e, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:37.988249 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c0edf-9a22-4b83-9aee-2340971d1df5 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:38.003360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24655388-2e8d-491f-8a50-03cd2f3d18a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:38.004056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24655388-2e8d-491f-8a50-03cd2f3d18a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.010571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 810/1606] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:37 2022] PUT /identity/v3/projects/d21ab544d0b547919794e3efb49ba0e9/users/67d86f09c6304121be2021cb5c15e63e/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:38.011883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c0edf-9a22-4b83-9aee-2340971d1df5 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:38.022573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24655388-2e8d-491f-8a50-03cd2f3d18a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.023948 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24655388-2e8d-491f-8a50-03cd2f3d18a7 admin admin] PATH_INFO: `/v3/projects/e17967a941be4a04bbc180e2cf3a7a55/users/53c0721dc18343b18993ff5af098841d/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.046327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24655388-2e8d-491f-8a50-03cd2f3d18a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e17967a941be4a04bbc180e2cf3a7a55, user_id=53c0721dc18343b18993ff5af098841d, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:38.049903 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24655388-2e8d-491f-8a50-03cd2f3d18a7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:38.078209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 797/1607] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:37 2022] PUT /identity/v3/projects/e17967a941be4a04bbc180e2cf3a7a55/users/53c0721dc18343b18993ff5af098841d/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:38.078394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24655388-2e8d-491f-8a50-03cd2f3d18a7 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:38.090846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c0edf-9a22-4b83-9aee-2340971d1df5 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:38.149883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22746673-f664-4ad7-a373-1e7d58d87d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:38.150509 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22746673-f664-4ad7-a373-1e7d58d87d2d service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.150816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22746673-f664-4ad7-a373-1e7d58d87d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.151086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22746673-f664-4ad7-a373-1e7d58d87d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.155014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24655388-2e8d-491f-8a50-03cd2f3d18a7 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:38.159095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3bcd6-85ba-4036-bcdd-f104fc01be80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:38.159959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3bcd6-85ba-4036-bcdd-f104fc01be80 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.160231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3bcd6-85ba-4036-bcdd-f104fc01be80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.160778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3bcd6-85ba-4036-bcdd-f104fc01be80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.214674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22746673-f664-4ad7-a373-1e7d58d87d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:38.216116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22746673-f664-4ad7-a373-1e7d58d87d2d service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:38.222818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3bcd6-85ba-4036-bcdd-f104fc01be80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:38.224419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3bcd6-85ba-4036-bcdd-f104fc01be80 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:38.225689 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 811/1608] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:38.226709 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22746673-f664-4ad7-a373-1e7d58d87d2d service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:38.234394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 798/1609] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:38.235825 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaabfe-23ec-43fc-95c2-028c074b1a0b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.236110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaabfe-23ec-43fc-95c2-028c074b1a0b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.236379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaabfe-23ec-43fc-95c2-028c074b1a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.236892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22746673-f664-4ad7-a373-1e7d58d87d2d service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:38.242236 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dde5b-4a1d-4c05-ba16-09ace198d527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:38.243089 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dde5b-4a1d-4c05-ba16-09ace198d527 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.243479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dde5b-4a1d-4c05-ba16-09ace198d527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.243827 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dde5b-4a1d-4c05-ba16-09ace198d527 admin admin] PATH_INFO: `/v3/projects/1f227ee89ce84dec84410a1db9b479fc/users/1ce1cd9c19584549b794df8e487abf00/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.262065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607dde5b-4a1d-4c05-ba16-09ace198d527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f227ee89ce84dec84410a1db9b479fc, user_id=1ce1cd9c19584549b794df8e487abf00, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:38.265614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607dde5b-4a1d-4c05-ba16-09ace198d527 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:38.268995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-dfdaabfe-23ec-43fc-95c2-028c074b1a0b None None] MFA Rules not processed for user `a89acf33520f4842ad411207309ebf9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:38.270458 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dfdaabfe-23ec-43fc-95c2-028c074b1a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:38.283675 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 812/1610] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:38 2022] PUT /identity/v3/projects/1f227ee89ce84dec84410a1db9b479fc/users/1ce1cd9c19584549b794df8e487abf00/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:38.285004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dde5b-4a1d-4c05-ba16-09ace198d527 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:38.296566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:38.333073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:38.362651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 799/1611] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:38.365271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd13db-cb65-43af-a71a-874c840c5d74 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.368331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd13db-cb65-43af-a71a-874c840c5d74 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.369783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd13db-cb65-43af-a71a-874c840c5d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.387671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dde5b-4a1d-4c05-ba16-09ace198d527 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:38.394850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df50b5-734c-4a09-9975-ee53a625a22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:38.395845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df50b5-734c-4a09-9975-ee53a625a22d admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.396088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df50b5-734c-4a09-9975-ee53a625a22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.396553 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df50b5-734c-4a09-9975-ee53a625a22d admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.397404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df50b5-734c-4a09-9975-ee53a625a22d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:38.399534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df50b5-734c-4a09-9975-ee53a625a22d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:38.422654 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-25fd13db-cb65-43af-a71a-874c840c5d74 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:38.425449 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-25fd13db-cb65-43af-a71a-874c840c5d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:38.438720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 800/1612] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:38.440010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 813/1613] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:38 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:38.440130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b748928-1dd3-490a-970b-70475ee8d9ed None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.440372 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b748928-1dd3-490a-970b-70475ee8d9ed None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.440729 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b748928-1dd3-490a-970b-70475ee8d9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.445111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c343e50-c09b-46f8-9714-a78a82c2d19d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.445447 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c343e50-c09b-46f8-9714-a78a82c2d19d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.445784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c343e50-c09b-46f8-9714-a78a82c2d19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.472960 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3b748928-1dd3-490a-970b-70475ee8d9ed None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:38.474324 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3b748928-1dd3-490a-970b-70475ee8d9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:38.480584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-0c343e50-c09b-46f8-9714-a78a82c2d19d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:38.481991 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0c343e50-c09b-46f8-9714-a78a82c2d19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:38.574736 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 814/1614] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:38.579410 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ca902-4125-4780-8645-08ce81978c2d None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.579786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ca902-4125-4780-8645-08ce81978c2d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.580250 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ca902-4125-4780-8645-08ce81978c2d None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.582525 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 815/1615] 10.222.0.23 () {54 vars in 962 bytes} [Fri Sep 9 16:31:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:31:38.583388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ca902-4125-4780-8645-08ce81978c2d None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:38.593761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 801/1616] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:38.595796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b748928-1dd3-490a-970b-70475ee8d9ed None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:38.600594 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ca902-4125-4780-8645-08ce81978c2d None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:38.608104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee49d5a-fa29-4a9f-a2dc-b32fbb771468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:38.608612 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee49d5a-fa29-4a9f-a2dc-b32fbb771468 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.609292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee49d5a-fa29-4a9f-a2dc-b32fbb771468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.609292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee49d5a-fa29-4a9f-a2dc-b32fbb771468 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.609897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee49d5a-fa29-4a9f-a2dc-b32fbb771468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:38.612687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee49d5a-fa29-4a9f-a2dc-b32fbb771468 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:38.652693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 816/1617] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:38 2022] POST /identity/v3/users => generated 293 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:38.653881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee49d5a-fa29-4a9f-a2dc-b32fbb771468 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:38.710272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b748928-1dd3-490a-970b-70475ee8d9ed None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:38.717225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3825550-f1ec-437d-8a13-3a04e8d0dd35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:38.718234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3825550-f1ec-437d-8a13-3a04e8d0dd35 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.718709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3825550-f1ec-437d-8a13-3a04e8d0dd35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.719152 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3825550-f1ec-437d-8a13-3a04e8d0dd35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.729322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee49d5a-fa29-4a9f-a2dc-b32fbb771468 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:38.733872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3825550-f1ec-437d-8a13-3a04e8d0dd35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:38.735400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352be90e-20c2-4c6e-a39f-7bde51bbc06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:38.735674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3825550-f1ec-437d-8a13-3a04e8d0dd35 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:38.736035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352be90e-20c2-4c6e-a39f-7bde51bbc06a service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.736313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352be90e-20c2-4c6e-a39f-7bde51bbc06a service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.736583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352be90e-20c2-4c6e-a39f-7bde51bbc06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.748396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352be90e-20c2-4c6e-a39f-7bde51bbc06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:38.750149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352be90e-20c2-4c6e-a39f-7bde51bbc06a service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:38.752960 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 802/1618] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:38.754302 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fde78-ea95-4a30-9dd1-a8664a0e9205 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.756537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fde78-ea95-4a30-9dd1-a8664a0e9205 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.756818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fde78-ea95-4a30-9dd1-a8664a0e9205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.763832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 817/1619] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:38.766422 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352be90e-20c2-4c6e-a39f-7bde51bbc06a service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:38.785160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352be90e-20c2-4c6e-a39f-7bde51bbc06a service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:38.795261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0835d-53bc-463f-8d22-bf0ffd5e0e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:38.796831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0835d-53bc-463f-8d22-bf0ffd5e0e96 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.797112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0835d-53bc-463f-8d22-bf0ffd5e0e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.798781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0835d-53bc-463f-8d22-bf0ffd5e0e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.821919 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0835d-53bc-463f-8d22-bf0ffd5e0e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:38.822861 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f46fde78-ea95-4a30-9dd1-a8664a0e9205 None None] MFA Rules not processed for user `152a41ad598b4b01b1881d928c53c848`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:38.824860 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0835d-53bc-463f-8d22-bf0ffd5e0e96 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:38.826423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f46fde78-ea95-4a30-9dd1-a8664a0e9205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:38.852636 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 818/1620] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:38.854078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0835d-53bc-463f-8d22-bf0ffd5e0e96 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:38.876303 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0835d-53bc-463f-8d22-bf0ffd5e0e96 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:38.891491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328f0e2-e145-4e32-839d-e56a26f33fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:38.893263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328f0e2-e145-4e32-839d-e56a26f33fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.893742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328f0e2-e145-4e32-839d-e56a26f33fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.894533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328f0e2-e145-4e32-839d-e56a26f33fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.897863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328f0e2-e145-4e32-839d-e56a26f33fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:38.901697 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328f0e2-e145-4e32-839d-e56a26f33fd1 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:38.906891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:38.932644 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 819/1621] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:38.937073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328f0e2-e145-4e32-839d-e56a26f33fd1 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:38.943600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 803/1622] 10.222.0.23 () {58 vars in 1059 bytes} [Fri Sep 9 16:31:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:38.952970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fde78-ea95-4a30-9dd1-a8664a0e9205 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:38.959158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328f0e2-e145-4e32-839d-e56a26f33fd1 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:38.965115 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6dfff-0d29-4efe-b0d5-71965bd7acb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:38.966356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6dfff-0d29-4efe-b0d5-71965bd7acb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:38.967202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6dfff-0d29-4efe-b0d5-71965bd7acb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:38.967845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6dfff-0d29-4efe-b0d5-71965bd7acb5 admin admin] PATH_INFO: `/v3/projects/9eb6c8a49195428abd22eca13da534e0/users/2cf2cef9de4244e59477f48993a2e2d6/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:38.991419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:38.995146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fde78-ea95-4a30-9dd1-a8664a0e9205 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:39.006712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a6dfff-0d29-4efe-b0d5-71965bd7acb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb6c8a49195428abd22eca13da534e0, user_id=2cf2cef9de4244e59477f48993a2e2d6, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:39.010348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab0121-7d6a-4df4-8520-b356ed644fa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:39.011015 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab0121-7d6a-4df4-8520-b356ed644fa7 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:39.011139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a6dfff-0d29-4efe-b0d5-71965bd7acb5 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:39.011374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab0121-7d6a-4df4-8520-b356ed644fa7 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:39.011548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab0121-7d6a-4df4-8520-b356ed644fa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:39.039559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 820/1623] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:38 2022] PUT /identity/v3/projects/9eb6c8a49195428abd22eca13da534e0/users/2cf2cef9de4244e59477f48993a2e2d6/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:39.044713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6dfff-0d29-4efe-b0d5-71965bd7acb5 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:39.150846 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab0121-7d6a-4df4-8520-b356ed644fa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:39.156029 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab0121-7d6a-4df4-8520-b356ed644fa7 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:39.191275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 804/1624] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:39.194408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6dfff-0d29-4efe-b0d5-71965bd7acb5 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:39.201778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501084f-d5a8-4385-bf45-33cb5379181f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:39.202634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501084f-d5a8-4385-bf45-33cb5379181f admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:39.202983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501084f-d5a8-4385-bf45-33cb5379181f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:39.203621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501084f-d5a8-4385-bf45-33cb5379181f admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:39.204447 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501084f-d5a8-4385-bf45-33cb5379181f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:39.207032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501084f-d5a8-4385-bf45-33cb5379181f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:39.224047 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 821/1625] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:39.225286 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab0121-7d6a-4df4-8520-b356ed644fa7 service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:39.241108 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab0121-7d6a-4df4-8520-b356ed644fa7 service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:39.253270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbaacb-e3ac-4571-9508-871c2e8dc9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:39.253790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbaacb-e3ac-4571-9508-871c2e8dc9a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:39.254075 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbaacb-e3ac-4571-9508-871c2e8dc9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:39.254432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbaacb-e3ac-4571-9508-871c2e8dc9a6 admin admin] PATH_INFO: `/v3/projects/9eb6c8a49195428abd22eca13da534e0/users/2cf2cef9de4244e59477f48993a2e2d6/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:39.258058 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbaacb-e3ac-4571-9508-871c2e8dc9a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb6c8a49195428abd22eca13da534e0, user_id=2cf2cef9de4244e59477f48993a2e2d6, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:39.260781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbaacb-e3ac-4571-9508-871c2e8dc9a6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:39.286349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 805/1626] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:39 2022] PUT /identity/v3/projects/9eb6c8a49195428abd22eca13da534e0/users/2cf2cef9de4244e59477f48993a2e2d6/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:31:39.297759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5958e92-429d-4fae-ac0e-3506f7417034 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:39.298101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5958e92-429d-4fae-ac0e-3506f7417034 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:39.298387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5958e92-429d-4fae-ac0e-3506f7417034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:39.363310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-b5958e92-429d-4fae-ac0e-3506f7417034 None None] MFA Rules not processed for user `2cf2cef9de4244e59477f48993a2e2d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:39.364609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b5958e92-429d-4fae-ac0e-3506f7417034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:39.464107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 822/1627] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:39.475542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbaacb-e3ac-4571-9508-871c2e8dc9a6 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:39.588713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788b75a-a56a-4692-985d-7adaf7b7d63e None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:39.589138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788b75a-a56a-4692-985d-7adaf7b7d63e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:39.589764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788b75a-a56a-4692-985d-7adaf7b7d63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:39.603394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbaacb-e3ac-4571-9508-871c2e8dc9a6 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:39.609268 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8163e2-e023-42ca-9ab7-57ac9b34e5f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:39.610102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8163e2-e023-42ca-9ab7-57ac9b34e5f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:39.610441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8163e2-e023-42ca-9ab7-57ac9b34e5f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:39.610726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8163e2-e023-42ca-9ab7-57ac9b34e5f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:39.624511 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8163e2-e023-42ca-9ab7-57ac9b34e5f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:39.628789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8163e2-e023-42ca-9ab7-57ac9b34e5f6 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:39.647489 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d788b75a-a56a-4692-985d-7adaf7b7d63e None None] MFA Rules not processed for user `00f532e0d03048cf86f9406d53e71d16`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:39.649205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 806/1628] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:39.654837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d788b75a-a56a-4692-985d-7adaf7b7d63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:39.776050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:39.789809 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 823/1629] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:39 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:39.795938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8163e2-e023-42ca-9ab7-57ac9b34e5f6 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:39.823508 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788b75a-a56a-4692-985d-7adaf7b7d63e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:39.836087 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788b75a-a56a-4692-985d-7adaf7b7d63e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:39.842674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089dd41a-65d8-4807-aa8c-b3acfc52cf16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:39.843472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dd41a-65d8-4807-aa8c-b3acfc52cf16 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:39.844178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dd41a-65d8-4807-aa8c-b3acfc52cf16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:39.844459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dd41a-65d8-4807-aa8c-b3acfc52cf16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:39.923526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8163e2-e023-42ca-9ab7-57ac9b34e5f6 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:39.931615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bbb11-e0e4-46d8-9559-c0e8bc1f4102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:39.932711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bbb11-e0e4-46d8-9559-c0e8bc1f4102 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:39.933217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bbb11-e0e4-46d8-9559-c0e8bc1f4102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:39.933700 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bbb11-e0e4-46d8-9559-c0e8bc1f4102 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:39.934589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bbb11-e0e4-46d8-9559-c0e8bc1f4102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:39.936844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bbb11-e0e4-46d8-9559-c0e8bc1f4102 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:39.950606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:39.977493 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 807/1630] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:39 2022] POST /identity/v3/projects => generated 403 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:39.979350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bbb11-e0e4-46d8-9559-c0e8bc1f4102 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:39.985723 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089dd41a-65d8-4807-aa8c-b3acfc52cf16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:39.987414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089dd41a-65d8-4807-aa8c-b3acfc52cf16 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:40.008629 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 824/1631] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:31:40.116080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bbb11-e0e4-46d8-9559-c0e8bc1f4102 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:40.123672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1341e-8f34-470b-b8c2-6305caf96852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:40.124600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1341e-8f34-470b-b8c2-6305caf96852 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:40.124744 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1341e-8f34-470b-b8c2-6305caf96852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:40.124963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1341e-8f34-470b-b8c2-6305caf96852 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:40.126010 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1341e-8f34-470b-b8c2-6305caf96852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:40.129034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1341e-8f34-470b-b8c2-6305caf96852 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:40.176499 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8e54b-8342-41de-959d-c00dee43d5fa None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:40.177123 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 808/1632] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:39 2022] POST /identity/v3/users => generated 308 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:40.177488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8e54b-8342-41de-959d-c00dee43d5fa None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:40.178000 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8e54b-8342-41de-959d-c00dee43d5fa None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:40.178900 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1341e-8f34-470b-b8c2-6305caf96852 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:40.180274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 825/1633] 10.222.0.23 () {54 vars in 962 bytes} [Fri Sep 9 16:31:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:31:40.200527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1341e-8f34-470b-b8c2-6305caf96852 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:40.201950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2aa34f-f920-470f-9b57-6b3ee044173c None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:40.202244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2aa34f-f920-470f-9b57-6b3ee044173c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:40.202709 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2aa34f-f920-470f-9b57-6b3ee044173c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:40.213112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cf8ba-ed8d-40de-8ad8-13d03ec55829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:40.215048 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cf8ba-ed8d-40de-8ad8-13d03ec55829 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:40.219923 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cf8ba-ed8d-40de-8ad8-13d03ec55829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:40.219923 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cf8ba-ed8d-40de-8ad8-13d03ec55829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:40.220973 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511cf8ba-ed8d-40de-8ad8-13d03ec55829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:40.226378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511cf8ba-ed8d-40de-8ad8-13d03ec55829 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:40.240098 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-be2aa34f-f920-470f-9b57-6b3ee044173c None None] MFA Rules not processed for user `152a41ad598b4b01b1881d928c53c848`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:40.242493 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-be2aa34f-f920-470f-9b57-6b3ee044173c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:40.264173 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 809/1634] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:40.265527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cf8ba-ed8d-40de-8ad8-13d03ec55829 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:40.302419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cf8ba-ed8d-40de-8ad8-13d03ec55829 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:40.331764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216535cf-e202-4a35-9a17-8ecfb4f03ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:40.332518 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216535cf-e202-4a35-9a17-8ecfb4f03ced admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:40.332804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216535cf-e202-4a35-9a17-8ecfb4f03ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:40.333369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216535cf-e202-4a35-9a17-8ecfb4f03ced admin admin] PATH_INFO: `/v3/projects/7ada5ae060964366b003f3a329d4b066/users/46e3bdbed41d44c8a2362b83fb6d25c9/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:40.370911 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 826/1635] 10.222.0.23 () {58 vars in 1059 bytes} [Fri Sep 9 16:31:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:31:40.372471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa26da-17c6-4084-95ca-1c46cfdee5da None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:40.373235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa26da-17c6-4084-95ca-1c46cfdee5da None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:40.373235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa26da-17c6-4084-95ca-1c46cfdee5da None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:40.379712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 827/1636] 10.222.0.23 () {54 vars in 912 bytes} [Fri Sep 9 16:31:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:31:40.381567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c0b31-4aa3-4321-aedd-eecedb3b3393 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:40.381920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c0b31-4aa3-4321-aedd-eecedb3b3393 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:40.382264 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c0b31-4aa3-4321-aedd-eecedb3b3393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:40.382975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216535cf-e202-4a35-9a17-8ecfb4f03ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ada5ae060964366b003f3a329d4b066, user_id=46e3bdbed41d44c8a2362b83fb6d25c9, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:40.388195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216535cf-e202-4a35-9a17-8ecfb4f03ced admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:40.415161 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-223c0b31-4aa3-4321-aedd-eecedb3b3393 None None] MFA Rules not processed for user `152a41ad598b4b01b1881d928c53c848`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:40.420489 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-223c0b31-4aa3-4321-aedd-eecedb3b3393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:40.429432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 810/1637] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:40 2022] PUT /identity/v3/projects/7ada5ae060964366b003f3a329d4b066/users/46e3bdbed41d44c8a2362b83fb6d25c9/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:40.441837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216535cf-e202-4a35-9a17-8ecfb4f03ced admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:40.642867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216535cf-e202-4a35-9a17-8ecfb4f03ced admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:40.642867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da963f96-6ede-488d-96bb-fdf49a11aa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:40.642867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da963f96-6ede-488d-96bb-fdf49a11aa23 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:40.642867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da963f96-6ede-488d-96bb-fdf49a11aa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:40.660732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da963f96-6ede-488d-96bb-fdf49a11aa23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:40.660732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da963f96-6ede-488d-96bb-fdf49a11aa23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:40.660732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da963f96-6ede-488d-96bb-fdf49a11aa23 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:40.660732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 811/1638] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:31:40.660732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3397a-02c0-4c67-b176-3d7fe3e5d5d1 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:40.660732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3397a-02c0-4c67-b176-3d7fe3e5d5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:40.661454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3397a-02c0-4c67-b176-3d7fe3e5d5d1 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:40.662545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 812/1639] 10.222.0.23 () {54 vars in 912 bytes} [Fri Sep 9 16:31:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:31:40.663490 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3397a-02c0-4c67-b176-3d7fe3e5d5d1 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:40.668523 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 828/1640] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:31:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:40.670919 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a894b03-07cc-407a-8146-f548da32e7a2 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:40.673360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a894b03-07cc-407a-8146-f548da32e7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:40.673742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a894b03-07cc-407a-8146-f548da32e7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:40.689553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3397a-02c0-4c67-b176-3d7fe3e5d5d1 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:40.699846 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714becea-b01c-49dc-9dcf-cc22fa108983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:40.701014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714becea-b01c-49dc-9dcf-cc22fa108983 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:40.701347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714becea-b01c-49dc-9dcf-cc22fa108983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:40.701690 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714becea-b01c-49dc-9dcf-cc22fa108983 admin admin] PATH_INFO: `/v3/projects/7ada5ae060964366b003f3a329d4b066/users/46e3bdbed41d44c8a2362b83fb6d25c9/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:40.724805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714becea-b01c-49dc-9dcf-cc22fa108983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ada5ae060964366b003f3a329d4b066, user_id=46e3bdbed41d44c8a2362b83fb6d25c9, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:40.728005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714becea-b01c-49dc-9dcf-cc22fa108983 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:40.733194 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-2a894b03-07cc-407a-8146-f548da32e7a2 None None] MFA Rules not processed for user `152a41ad598b4b01b1881d928c53c848`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:40.734711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2a894b03-07cc-407a-8146-f548da32e7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:40.747967 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 813/1641] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:40 2022] PUT /identity/v3/projects/7ada5ae060964366b003f3a329d4b066/users/46e3bdbed41d44c8a2362b83fb6d25c9/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:40.753393 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714becea-b01c-49dc-9dcf-cc22fa108983 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:40.759463 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 829/1642] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:31:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:40.768025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a894b03-07cc-407a-8146-f548da32e7a2 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:40.800792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:40.804237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:40.839857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714becea-b01c-49dc-9dcf-cc22fa108983 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:40.845382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82c90b-7b3d-4654-9876-557d0e47781d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:40.846046 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82c90b-7b3d-4654-9876-557d0e47781d service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:40.846327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82c90b-7b3d-4654-9876-557d0e47781d service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:40.846627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82c90b-7b3d-4654-9876-557d0e47781d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:40.869899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a894b03-07cc-407a-8146-f548da32e7a2 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:40.875197 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3eb18-f3a2-4aeb-8701-970b36ab2763 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:40.876266 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3eb18-f3a2-4aeb-8701-970b36ab2763 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:40.876841 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3eb18-f3a2-4aeb-8701-970b36ab2763 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:40.877300 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3eb18-f3a2-4aeb-8701-970b36ab2763 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:40.982809 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3eb18-f3a2-4aeb-8701-970b36ab2763 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:40.985039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3eb18-f3a2-4aeb-8701-970b36ab2763 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:41.004913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82c90b-7b3d-4654-9876-557d0e47781d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:41.006578 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82c90b-7b3d-4654-9876-557d0e47781d service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:41.020713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 830/1643] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:41.024631 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 814/1644] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:41.270239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3eb18-f3a2-4aeb-8701-970b36ab2763 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:41.427923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3eb18-f3a2-4aeb-8701-970b36ab2763 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:41.438597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85beb21e-e8cd-458f-982b-77dedddaa83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:41.442378 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85beb21e-e8cd-458f-982b-77dedddaa83c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:41.443021 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85beb21e-e8cd-458f-982b-77dedddaa83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:41.443936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85beb21e-e8cd-458f-982b-77dedddaa83c admin admin] PATH_INFO: `/v3/users/a89acf33520f4842ad411207309ebf9e` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:41.448308 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85beb21e-e8cd-458f-982b-77dedddaa83c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a89acf33520f4842ad411207309ebf9e)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:41.453363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85beb21e-e8cd-458f-982b-77dedddaa83c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:41.585348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-85beb21e-e8cd-458f-982b-77dedddaa83c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a89acf33520f4842ad411207309ebf9e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:41.586218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-85beb21e-e8cd-458f-982b-77dedddaa83c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a89acf33520f4842ad411207309ebf9e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:41.588225 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-85beb21e-e8cd-458f-982b-77dedddaa83c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a89acf33520f4842ad411207309ebf9e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:41.611867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-85beb21e-e8cd-458f-982b-77dedddaa83c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a89acf33520f4842ad411207309ebf9e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:41.614891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:41.643215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-85beb21e-e8cd-458f-982b-77dedddaa83c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a89acf33520f4842ad411207309ebf9e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:41.713118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2158d2-b2db-4d45-bd68-8e2747105395 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:41.713542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2158d2-b2db-4d45-bd68-8e2747105395 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:41.714675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2158d2-b2db-4d45-bd68-8e2747105395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:41.718600 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 831/1645] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:31:41 2022] DELETE /identity/v3/users/a89acf33520f4842ad411207309ebf9e => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:41.767255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8e2158d2-b2db-4d45-bd68-8e2747105395 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:41.767255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8e2158d2-b2db-4d45-bd68-8e2747105395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:41.900669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 815/1646] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:41.918224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85beb21e-e8cd-458f-982b-77dedddaa83c admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:41.919281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-85beb21e-e8cd-458f-982b-77dedddaa83c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:42.077110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85beb21e-e8cd-458f-982b-77dedddaa83c admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:42.095060 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5502293-f7ad-42e7-8059-be57971f4615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:42.095825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5502293-f7ad-42e7-8059-be57971f4615 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:42.097006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5502293-f7ad-42e7-8059-be57971f4615 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:42.098466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5502293-f7ad-42e7-8059-be57971f4615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:42.123280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5502293-f7ad-42e7-8059-be57971f4615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:42.124845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5502293-f7ad-42e7-8059-be57971f4615 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:42.142795 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 832/1647] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:43.215774 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2158d2-b2db-4d45-bd68-8e2747105395 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:43.217332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8e2158d2-b2db-4d45-bd68-8e2747105395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:43.230677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2158d2-b2db-4d45-bd68-8e2747105395 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:43.249085 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d9e58-86f4-411c-972f-d21e2dccce30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:43.250067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d9e58-86f4-411c-972f-d21e2dccce30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:43.250558 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d9e58-86f4-411c-972f-d21e2dccce30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:43.251018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d9e58-86f4-411c-972f-d21e2dccce30 admin admin] PATH_INFO: `/v3/projects/75e4208363cf47dfb52c9a5313068435` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:43.252699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d9e58-86f4-411c-972f-d21e2dccce30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75e4208363cf47dfb52c9a5313068435)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:43.255032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d9e58-86f4-411c-972f-d21e2dccce30 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:43.350549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cb9d9e58-86f4-411c-972f-d21e2dccce30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75e4208363cf47dfb52c9a5313068435', 'request_id': 'req-cb9d9e58-86f4-411c-972f-d21e2dccce30'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:43.358121 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cb9d9e58-86f4-411c-972f-d21e2dccce30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75e4208363cf47dfb52c9a5313068435', 'request_id': 'req-cb9d9e58-86f4-411c-972f-d21e2dccce30'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:43.370871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cb9d9e58-86f4-411c-972f-d21e2dccce30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75e4208363cf47dfb52c9a5313068435', 'request_id': 'req-cb9d9e58-86f4-411c-972f-d21e2dccce30'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:43.377503 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:43.404761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cb9d9e58-86f4-411c-972f-d21e2dccce30 admin admin] The token cache is being invalidate because project 75e4208363cf47dfb52c9a5313068435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:31:43.405067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cb9d9e58-86f4-411c-972f-d21e2dccce30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:43.405740 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cb9d9e58-86f4-411c-972f-d21e2dccce30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:43.408139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 816/1648] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:31:43 2022] DELETE /identity/v3/projects/75e4208363cf47dfb52c9a5313068435 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:43.414069 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35a168-6af0-439e-aac4-77d5dcfa7348 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:43.415440 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35a168-6af0-439e-aac4-77d5dcfa7348 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:43.416380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35a168-6af0-439e-aac4-77d5dcfa7348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:43.476730 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-4c35a168-6af0-439e-aac4-77d5dcfa7348 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:43.481858 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-4c35a168-6af0-439e-aac4-77d5dcfa7348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:43.692340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 833/1649] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:43.700055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2076b94-e0e2-4813-8839-2b4c1eb73464 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:43.700606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2076b94-e0e2-4813-8839-2b4c1eb73464 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:43.701078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2076b94-e0e2-4813-8839-2b4c1eb73464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:43.763721 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f2076b94-e0e2-4813-8839-2b4c1eb73464 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:43.766090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f2076b94-e0e2-4813-8839-2b4c1eb73464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:43.777398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 817/1650] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:43.781327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35a168-6af0-439e-aac4-77d5dcfa7348 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:43.794364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35a168-6af0-439e-aac4-77d5dcfa7348 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:43.802336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0094f0-1887-449c-871c-674153a2fe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:43.804362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0094f0-1887-449c-871c-674153a2fe07 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:43.804811 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0094f0-1887-449c-871c-674153a2fe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:43.805197 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0094f0-1887-449c-871c-674153a2fe07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:43.805975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0094f0-1887-449c-871c-674153a2fe07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:31:43.806393 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0094f0-1887-449c-871c-674153a2fe07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:43.808235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0094f0-1887-449c-871c-674153a2fe07 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:43.845681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 834/1651] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:31:43.851306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2076b94-e0e2-4813-8839-2b4c1eb73464 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:43.865511 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2076b94-e0e2-4813-8839-2b4c1eb73464 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:43.874878 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e6de4-bd85-4811-aede-4bdf42096873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:43.875633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e6de4-bd85-4811-aede-4bdf42096873 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:43.875768 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e6de4-bd85-4811-aede-4bdf42096873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:43.878159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e6de4-bd85-4811-aede-4bdf42096873 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:43.880790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e6de4-bd85-4811-aede-4bdf42096873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:43.884816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e6de4-bd85-4811-aede-4bdf42096873 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:43.950692 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 818/1652] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:43 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:43.955749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0094f0-1887-449c-871c-674153a2fe07 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:43.978556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0094f0-1887-449c-871c-674153a2fe07 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:44.140506 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfdeff-d05d-4bec-9db0-8827ffaf8da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:44.141195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfdeff-d05d-4bec-9db0-8827ffaf8da3 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:44.141647 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfdeff-d05d-4bec-9db0-8827ffaf8da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:44.141964 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfdeff-d05d-4bec-9db0-8827ffaf8da3 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:44.142795 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbfdeff-d05d-4bec-9db0-8827ffaf8da3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:44.145131 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbfdeff-d05d-4bec-9db0-8827ffaf8da3 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:44.175967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:44.195338 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b79003-69c8-465e-98e9-dc802caf0987 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:44.196046 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b79003-69c8-465e-98e9-dc802caf0987 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:44.196565 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b79003-69c8-465e-98e9-dc802caf0987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:44.201051 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 835/1653] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:43 2022] POST /identity/v3/users => generated 290 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:44.205550 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfdeff-d05d-4bec-9db0-8827ffaf8da3 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:44.225587 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfdeff-d05d-4bec-9db0-8827ffaf8da3 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:44.243409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89678188-0ff0-487c-ba6b-d40ee0c8a8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:44.244061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89678188-0ff0-487c-ba6b-d40ee0c8a8cc admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:44.244348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89678188-0ff0-487c-ba6b-d40ee0c8a8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:44.244621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89678188-0ff0-487c-ba6b-d40ee0c8a8cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:44.245435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89678188-0ff0-487c-ba6b-d40ee0c8a8cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:44.247259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89678188-0ff0-487c-ba6b-d40ee0c8a8cc admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:44.265779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 836/1654] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:31:44.269416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-99b79003-69c8-465e-98e9-dc802caf0987 None None] MFA Rules not processed for user `46e3bdbed41d44c8a2362b83fb6d25c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:44.269909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89678188-0ff0-487c-ba6b-d40ee0c8a8cc admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:44.272667 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-99b79003-69c8-465e-98e9-dc802caf0987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:44.290528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89678188-0ff0-487c-ba6b-d40ee0c8a8cc admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:44.304126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5f1af-e99d-441c-a08e-78b25dc2ed41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:44.305566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5f1af-e99d-441c-a08e-78b25dc2ed41 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:44.306236 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5f1af-e99d-441c-a08e-78b25dc2ed41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:44.306885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5f1af-e99d-441c-a08e-78b25dc2ed41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:44.308882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5f1af-e99d-441c-a08e-78b25dc2ed41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:44.311057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5f1af-e99d-441c-a08e-78b25dc2ed41 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:44.327561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 837/1655] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:44.331282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5f1af-e99d-441c-a08e-78b25dc2ed41 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:44.343932 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5f1af-e99d-441c-a08e-78b25dc2ed41 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:44.357124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96dadb-a043-4b0a-9978-698060620ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:44.358195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96dadb-a043-4b0a-9978-698060620ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:44.358718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96dadb-a043-4b0a-9978-698060620ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:44.359177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96dadb-a043-4b0a-9978-698060620ba2 admin admin] PATH_INFO: `/v3/projects/d92029b617014ba38e68e2a41f323d34/users/7b9a0505a23d4a58a8453922fc0fc397/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:44.386438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96dadb-a043-4b0a-9978-698060620ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d92029b617014ba38e68e2a41f323d34, user_id=7b9a0505a23d4a58a8453922fc0fc397, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:44.390026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96dadb-a043-4b0a-9978-698060620ba2 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:44.412187 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 838/1656] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:44 2022] PUT /identity/v3/projects/d92029b617014ba38e68e2a41f323d34/users/7b9a0505a23d4a58a8453922fc0fc397/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:31:44.424422 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22f7f5-6604-42be-b742-74aad4c77532 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:44.425788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22f7f5-6604-42be-b742-74aad4c77532 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:44.427051 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22f7f5-6604-42be-b742-74aad4c77532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:44.461314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 819/1657] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:44 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:44.471990 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-5d22f7f5-6604-42be-b742-74aad4c77532 None None] MFA Rules not processed for user `7b9a0505a23d4a58a8453922fc0fc397`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:44.474207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5d22f7f5-6604-42be-b742-74aad4c77532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:44.580625 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 839/1658] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:44.585270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b79003-69c8-465e-98e9-dc802caf0987 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:44.598473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b79003-69c8-465e-98e9-dc802caf0987 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:44.719293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb2c9a-730a-4bcc-ae2a-8d0008c6b017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:44.719956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb2c9a-730a-4bcc-ae2a-8d0008c6b017 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:44.720283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb2c9a-730a-4bcc-ae2a-8d0008c6b017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:44.720554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb2c9a-730a-4bcc-ae2a-8d0008c6b017 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:44.721196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb2c9a-730a-4bcc-ae2a-8d0008c6b017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:44.723293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb2c9a-730a-4bcc-ae2a-8d0008c6b017 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:44.787678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 820/1659] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:44 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:44.788916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22f7f5-6604-42be-b742-74aad4c77532 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:44.816229 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22f7f5-6604-42be-b742-74aad4c77532 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:44.952528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29a22b-55a5-4ca9-9d49-324f74b391bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:44.953381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29a22b-55a5-4ca9-9d49-324f74b391bc admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:44.954072 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29a22b-55a5-4ca9-9d49-324f74b391bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:44.954346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29a22b-55a5-4ca9-9d49-324f74b391bc admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:44.958107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc29a22b-55a5-4ca9-9d49-324f74b391bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:44.963111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc29a22b-55a5-4ca9-9d49-324f74b391bc admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:45.015269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 840/1660] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:44 2022] POST /identity/v3/users => generated 296 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:45.019051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb2c9a-730a-4bcc-ae2a-8d0008c6b017 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:45.051430 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb2c9a-730a-4bcc-ae2a-8d0008c6b017 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:45.064695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297a20b-d920-4ecc-baab-4eb6d12cae52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:45.066206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297a20b-d920-4ecc-baab-4eb6d12cae52 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:45.066206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297a20b-d920-4ecc-baab-4eb6d12cae52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:45.066206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297a20b-d920-4ecc-baab-4eb6d12cae52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:45.069841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297a20b-d920-4ecc-baab-4eb6d12cae52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:45.087923 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297a20b-d920-4ecc-baab-4eb6d12cae52 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:45.105384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 821/1661] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:45.111279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29a22b-55a5-4ca9-9d49-324f74b391bc admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:45.140852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29a22b-55a5-4ca9-9d49-324f74b391bc admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:45.147217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e846eb-c9a5-4dfb-8128-63eacd767529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:45.148297 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e846eb-c9a5-4dfb-8128-63eacd767529 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:45.148672 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e846eb-c9a5-4dfb-8128-63eacd767529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:45.149796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e846eb-c9a5-4dfb-8128-63eacd767529 admin admin] PATH_INFO: `/v3/projects/7a55eb91cec2492d96b7547af043e387/users/09ca60a1ac6f435c921e6c578aaa1d87/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:45.176949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e846eb-c9a5-4dfb-8128-63eacd767529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a55eb91cec2492d96b7547af043e387, user_id=09ca60a1ac6f435c921e6c578aaa1d87, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:45.182171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e846eb-c9a5-4dfb-8128-63eacd767529 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:45.214741 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 841/1662] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:45 2022] PUT /identity/v3/projects/7a55eb91cec2492d96b7547af043e387/users/09ca60a1ac6f435c921e6c578aaa1d87/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:45.220129 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297a20b-d920-4ecc-baab-4eb6d12cae52 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:45.242817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297a20b-d920-4ecc-baab-4eb6d12cae52 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:45.298772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:45.392516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d1886-8968-4bb9-9a34-bfe1a1d7d533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:45.393202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d1886-8968-4bb9-9a34-bfe1a1d7d533 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:45.393480 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d1886-8968-4bb9-9a34-bfe1a1d7d533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:45.393791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d1886-8968-4bb9-9a34-bfe1a1d7d533 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:45.394650 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d1886-8968-4bb9-9a34-bfe1a1d7d533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:45.397305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d1886-8968-4bb9-9a34-bfe1a1d7d533 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:45.415880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 822/1663] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:45.419436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e846eb-c9a5-4dfb-8128-63eacd767529 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:45.428363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:45.434700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e846eb-c9a5-4dfb-8128-63eacd767529 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:45.443535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b1af0-f875-4c17-a650-1bf7fe6ed03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:45.444781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b1af0-f875-4c17-a650-1bf7fe6ed03f admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:45.445257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b1af0-f875-4c17-a650-1bf7fe6ed03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:45.447063 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b1af0-f875-4c17-a650-1bf7fe6ed03f admin admin] PATH_INFO: `/v3/projects/7a55eb91cec2492d96b7547af043e387/users/09ca60a1ac6f435c921e6c578aaa1d87/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:45.451139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b1af0-f875-4c17-a650-1bf7fe6ed03f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a55eb91cec2492d96b7547af043e387, user_id=09ca60a1ac6f435c921e6c578aaa1d87, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:45.454384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b1af0-f875-4c17-a650-1bf7fe6ed03f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:45.475464 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 842/1664] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:45 2022] PUT /identity/v3/projects/7a55eb91cec2492d96b7547af043e387/users/09ca60a1ac6f435c921e6c578aaa1d87/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:45.486237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a5afb-c19a-46cd-b0f2-ea703e93fe7c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:45.486586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a5afb-c19a-46cd-b0f2-ea703e93fe7c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:45.487051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a5afb-c19a-46cd-b0f2-ea703e93fe7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:45.546275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-101a5afb-c19a-46cd-b0f2-ea703e93fe7c None None] MFA Rules not processed for user `09ca60a1ac6f435c921e6c578aaa1d87`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:45.548509 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-101a5afb-c19a-46cd-b0f2-ea703e93fe7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:45.725524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 823/1665] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:45.740974 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b1af0-f875-4c17-a650-1bf7fe6ed03f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:45.744007 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-084b1af0-f875-4c17-a650-1bf7fe6ed03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:45.878796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b1af0-f875-4c17-a650-1bf7fe6ed03f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:45.891342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91eb1a3-4e59-4183-b747-c064a1ad8cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:45.891915 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91eb1a3-4e59-4183-b747-c064a1ad8cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:45.892192 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91eb1a3-4e59-4183-b747-c064a1ad8cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:45.892471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91eb1a3-4e59-4183-b747-c064a1ad8cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:45.928923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91eb1a3-4e59-4183-b747-c064a1ad8cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:45.933607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91eb1a3-4e59-4183-b747-c064a1ad8cf2 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:45.965413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 843/1666] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:46.617917 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a5afb-c19a-46cd-b0f2-ea703e93fe7c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:46.633101 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a5afb-c19a-46cd-b0f2-ea703e93fe7c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:46.670380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2aa78-8930-49c3-ab8a-f1b0cec8ce2f tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09ca60a1ac6f435c921e6c578aaa1d87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a55eb91cec2492d96b7547af043e387', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:46.671087 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2aa78-8930-49c3-ab8a-f1b0cec8ce2f tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:46.671370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2aa78-8930-49c3-ab8a-f1b0cec8ce2f tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:46.671647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2aa78-8930-49c3-ab8a-f1b0cec8ce2f tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:46.672316 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2aa78-8930-49c3-ab8a-f1b0cec8ce2f tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:31:46.672618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2aa78-8930-49c3-ab8a-f1b0cec8ce2f tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:46.674240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2aa78-8930-49c3-ab8a-f1b0cec8ce2f tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:46.704580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 824/1667] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:31:46.709150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91eb1a3-4e59-4183-b747-c064a1ad8cf2 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:46.732382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91eb1a3-4e59-4183-b747-c064a1ad8cf2 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:46.739837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49da04-73f1-4260-a10f-c64e01e26c7e tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09ca60a1ac6f435c921e6c578aaa1d87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a55eb91cec2492d96b7547af043e387', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:46.740837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49da04-73f1-4260-a10f-c64e01e26c7e tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:46.741400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49da04-73f1-4260-a10f-c64e01e26c7e tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:46.741866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49da04-73f1-4260-a10f-c64e01e26c7e tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:46.742742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49da04-73f1-4260-a10f-c64e01e26c7e tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:46.748429 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49da04-73f1-4260-a10f-c64e01e26c7e tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:46.816232 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 844/1668] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:46 2022] POST /identity/v3/projects => generated 355 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:46.819939 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2aa78-8930-49c3-ab8a-f1b0cec8ce2f tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:46.833773 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2aa78-8930-49c3-ab8a-f1b0cec8ce2f tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:47.025431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa80cf-2a8b-4534-8499-4fee86be2acc tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09ca60a1ac6f435c921e6c578aaa1d87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a55eb91cec2492d96b7547af043e387', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:47.026365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa80cf-2a8b-4534-8499-4fee86be2acc tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:47.028283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa80cf-2a8b-4534-8499-4fee86be2acc tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:47.028781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa80cf-2a8b-4534-8499-4fee86be2acc tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:47.029733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa80cf-2a8b-4534-8499-4fee86be2acc tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:31:47.034545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa80cf-2a8b-4534-8499-4fee86be2acc tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:47.036462 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa80cf-2a8b-4534-8499-4fee86be2acc tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:47.091954 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 825/1669] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:31:47.238676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49da04-73f1-4260-a10f-c64e01e26c7e tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:47.254346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49da04-73f1-4260-a10f-c64e01e26c7e tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:47.269820 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0c835-5b84-49ea-a675-32c89d9e75ab tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09ca60a1ac6f435c921e6c578aaa1d87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a55eb91cec2492d96b7547af043e387', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:47.270540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0c835-5b84-49ea-a675-32c89d9e75ab tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:47.270826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0c835-5b84-49ea-a675-32c89d9e75ab tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:47.271105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0c835-5b84-49ea-a675-32c89d9e75ab tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] PATH_INFO: `/v3/projects/5957db52bd76401a8e1f01b8eed2a165` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:47.273161 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0c835-5b84-49ea-a675-32c89d9e75ab tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5957db52bd76401a8e1f01b8eed2a165)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:47.276431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: WARNING py.warnings [None req-beb0c835-5b84-49ea-a675-32c89d9e75ab tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:31:47.276431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: warnings.warn(msg) Sep 09 16:31:47.276431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: Sep 09 16:31:47.279773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0c835-5b84-49ea-a675-32c89d9e75ab tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:47.428833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-beb0c835-5b84-49ea-a675-32c89d9e75ab tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5957db52bd76401a8e1f01b8eed2a165', 'request_id': 'req-beb0c835-5b84-49ea-a675-32c89d9e75ab'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:47.480584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-beb0c835-5b84-49ea-a675-32c89d9e75ab tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5957db52bd76401a8e1f01b8eed2a165', 'request_id': 'req-beb0c835-5b84-49ea-a675-32c89d9e75ab'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:47.502876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-beb0c835-5b84-49ea-a675-32c89d9e75ab tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5957db52bd76401a8e1f01b8eed2a165', 'request_id': 'req-beb0c835-5b84-49ea-a675-32c89d9e75ab'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:47.525932 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-beb0c835-5b84-49ea-a675-32c89d9e75ab tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] The token cache is being invalidate because project 5957db52bd76401a8e1f01b8eed2a165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:31:47.526784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-beb0c835-5b84-49ea-a675-32c89d9e75ab tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:47.527616 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-beb0c835-5b84-49ea-a675-32c89d9e75ab tempest-VolumeQuotaClassesTest-751823243 tempest-VolumeQuotaClassesTest-751823243-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:47.530150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 845/1670] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:31:47 2022] DELETE /identity/v3/projects/5957db52bd76401a8e1f01b8eed2a165 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:48.108687 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd079b-036c-4263-84ec-e40b60213436 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:48.110188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd079b-036c-4263-84ec-e40b60213436 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:48.110947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd079b-036c-4263-84ec-e40b60213436 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:48.115377 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 826/1671] 10.222.0.23 () {54 vars in 921 bytes} [Fri Sep 9 16:31:48 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:31:48.199521 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded362cd-0f87-474f-8f4f-73e1e199fcbe None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:48.200671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded362cd-0f87-474f-8f4f-73e1e199fcbe None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:48.201112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded362cd-0f87-474f-8f4f-73e1e199fcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:48.254005 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-ded362cd-0f87-474f-8f4f-73e1e199fcbe None None] MFA Rules not processed for user `784b6147de91435292e4cf3940bcad9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:48.256223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ded362cd-0f87-474f-8f4f-73e1e199fcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:48.283939 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd079b-036c-4263-84ec-e40b60213436 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:48.285983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c9dd079b-036c-4263-84ec-e40b60213436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:48.386851 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:48.403632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:48.487984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 846/1672] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Sep 9 16:31:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:48.503392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd079b-036c-4263-84ec-e40b60213436 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:48.525032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:48.526426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:48.528169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:48.528539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] PATH_INFO: `/v3/users/7b9a0505a23d4a58a8453922fc0fc397` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:48.532705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b9a0505a23d4a58a8453922fc0fc397)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:48.533867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: WARNING py.warnings [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 16:31:48.533867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: warnings.warn(msg) Sep 09 16:31:48.533867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: Sep 09 16:31:48.535801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:48.727029 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b9a0505a23d4a58a8453922fc0fc397'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:48.730286 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b9a0505a23d4a58a8453922fc0fc397'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:48.731413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b9a0505a23d4a58a8453922fc0fc397'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:48.752517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded362cd-0f87-474f-8f4f-73e1e199fcbe None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:48.753983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ded362cd-0f87-474f-8f4f-73e1e199fcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:48.761206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b9a0505a23d4a58a8453922fc0fc397'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:48.794798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded362cd-0f87-474f-8f4f-73e1e199fcbe None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:48.801680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a0337-a06f-47f6-ab11-569e65ab25da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784b6147de91435292e4cf3940bcad9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:48.802309 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a0337-a06f-47f6-ab11-569e65ab25da service placement] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:48.802621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a0337-a06f-47f6-ab11-569e65ab25da service placement] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:48.802893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a0337-a06f-47f6-ab11-569e65ab25da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:48.805115 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-725a0337-a06f-47f6-ab11-569e65ab25da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:48.820073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a0337-a06f-47f6-ab11-569e65ab25da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:48.821685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a0337-a06f-47f6-ab11-569e65ab25da service placement] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:48.827072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b9a0505a23d4a58a8453922fc0fc397'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:48.846689 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 847/1673] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:31:48.868277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 827/1674] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:31:48 2022] DELETE /identity/v3/users/7b9a0505a23d4a58a8453922fc0fc397 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:31:48.870225 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26176c54-fa51-4449-9647-d99afd1d082e None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:48.870542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26176c54-fa51-4449-9647-d99afd1d082e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:48.870912 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26176c54-fa51-4449-9647-d99afd1d082e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:48.906292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-26176c54-fa51-4449-9647-d99afd1d082e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:48.908320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-26176c54-fa51-4449-9647-d99afd1d082e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:49.137702 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 848/1675] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:31:49.247519 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:49.250368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:49.429309 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebf4f6-7f8c-44fb-a1ae-28dda784dfd2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:49.434254 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061b678-6e2d-4627-a6a9-d22170176e15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:49.434874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061b678-6e2d-4627-a6a9-d22170176e15 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:49.435156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061b678-6e2d-4627-a6a9-d22170176e15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:49.435723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061b678-6e2d-4627-a6a9-d22170176e15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:49.459533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061b678-6e2d-4627-a6a9-d22170176e15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:49.461402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061b678-6e2d-4627-a6a9-d22170176e15 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:49.489783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 828/1676] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:50.947269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26176c54-fa51-4449-9647-d99afd1d082e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:50.948752 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-26176c54-fa51-4449-9647-d99afd1d082e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:50.973056 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26176c54-fa51-4449-9647-d99afd1d082e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:50.979771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d81081-ef7a-46b6-8bd3-2da179319160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:50.980576 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d81081-ef7a-46b6-8bd3-2da179319160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:50.980865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d81081-ef7a-46b6-8bd3-2da179319160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:50.981141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d81081-ef7a-46b6-8bd3-2da179319160 admin admin] PATH_INFO: `/v3/users/b253af9b45d146c2bd87d64576fce8dc` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:50.984743 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d81081-ef7a-46b6-8bd3-2da179319160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b253af9b45d146c2bd87d64576fce8dc)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:50.986761 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d81081-ef7a-46b6-8bd3-2da179319160 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:51.089627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-41d81081-ef7a-46b6-8bd3-2da179319160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b253af9b45d146c2bd87d64576fce8dc'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.095030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-41d81081-ef7a-46b6-8bd3-2da179319160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b253af9b45d146c2bd87d64576fce8dc'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.096749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-41d81081-ef7a-46b6-8bd3-2da179319160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b253af9b45d146c2bd87d64576fce8dc'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.111590 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061b678-6e2d-4627-a6a9-d22170176e15 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:51.114220 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7061b678-6e2d-4627-a6a9-d22170176e15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:51.133317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-41d81081-ef7a-46b6-8bd3-2da179319160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b253af9b45d146c2bd87d64576fce8dc'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.143666 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061b678-6e2d-4627-a6a9-d22170176e15 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:51.157678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:51.159327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:51.159886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:51.160894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] PATH_INFO: `/v3/projects/d92029b617014ba38e68e2a41f323d34` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:51.165407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d92029b617014ba38e68e2a41f323d34)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:51.168453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-41d81081-ef7a-46b6-8bd3-2da179319160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b253af9b45d146c2bd87d64576fce8dc'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.173384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:51.225524 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 849/1677] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:31:50 2022] DELETE /identity/v3/users/b253af9b45d146c2bd87d64576fce8dc => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:51.228695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115efac7-b772-4d11-8228-bad2f7068615 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:51.228981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115efac7-b772-4d11-8228-bad2f7068615 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:51.230630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115efac7-b772-4d11-8228-bad2f7068615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:51.284552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-115efac7-b772-4d11-8228-bad2f7068615 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:51.286767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-115efac7-b772-4d11-8228-bad2f7068615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:51.290457 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:51.303980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd92029b617014ba38e68e2a41f323d34', 'request_id': 'req-2da5d8ef-b838-4b91-b620-3420a037da0a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.317404 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd92029b617014ba38e68e2a41f323d34', 'request_id': 'req-2da5d8ef-b838-4b91-b620-3420a037da0a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.334993 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd92029b617014ba38e68e2a41f323d34', 'request_id': 'req-2da5d8ef-b838-4b91-b620-3420a037da0a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.360025 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] The token cache is being invalidate because project d92029b617014ba38e68e2a41f323d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:31:51.360510 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.361672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.371940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 829/1678] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:31:51 2022] DELETE /identity/v3/projects/d92029b617014ba38e68e2a41f323d34 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:31:51.376608 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:51.378321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:51.389575 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:51.428360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 850/1679] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:51.526284 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5d8ef-b838-4b91-b620-3420a037da0a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:51.544721 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652d8bb-ef17-43fa-8a6e-803dcc44a4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:51.545404 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652d8bb-ef17-43fa-8a6e-803dcc44a4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:51.545685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652d8bb-ef17-43fa-8a6e-803dcc44a4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:51.545957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652d8bb-ef17-43fa-8a6e-803dcc44a4f1 admin admin] PATH_INFO: `/v3/users/09ca60a1ac6f435c921e6c578aaa1d87` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:51.547321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652d8bb-ef17-43fa-8a6e-803dcc44a4f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ca60a1ac6f435c921e6c578aaa1d87)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:51.549326 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652d8bb-ef17-43fa-8a6e-803dcc44a4f1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:51.574333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115efac7-b772-4d11-8228-bad2f7068615 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:51.575994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-115efac7-b772-4d11-8228-bad2f7068615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:51.633558 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2652d8bb-ef17-43fa-8a6e-803dcc44a4f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ca60a1ac6f435c921e6c578aaa1d87'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.634801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2652d8bb-ef17-43fa-8a6e-803dcc44a4f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ca60a1ac6f435c921e6c578aaa1d87'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.635861 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2652d8bb-ef17-43fa-8a6e-803dcc44a4f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ca60a1ac6f435c921e6c578aaa1d87'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.648657 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2652d8bb-ef17-43fa-8a6e-803dcc44a4f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ca60a1ac6f435c921e6c578aaa1d87'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.676266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2652d8bb-ef17-43fa-8a6e-803dcc44a4f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ca60a1ac6f435c921e6c578aaa1d87'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:51.696390 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115efac7-b772-4d11-8228-bad2f7068615 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:51.709505 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae066c-4edd-47ce-a671-138a802f0944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:51.710157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae066c-4edd-47ce-a671-138a802f0944 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:51.710436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae066c-4edd-47ce-a671-138a802f0944 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:51.710734 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae066c-4edd-47ce-a671-138a802f0944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:51.712559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7eae066c-4edd-47ce-a671-138a802f0944 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:51.714035 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 830/1680] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:31:51 2022] DELETE /identity/v3/users/09ca60a1ac6f435c921e6c578aaa1d87 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:31:51.733172 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae066c-4edd-47ce-a671-138a802f0944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:51.738235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae066c-4edd-47ce-a671-138a802f0944 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:51.765582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 851/1681] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:52.117400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add397f9-3ff0-4912-b200-72a8eacf4b70 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:52.118316 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add397f9-3ff0-4912-b200-72a8eacf4b70 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:52.118727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add397f9-3ff0-4912-b200-72a8eacf4b70 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:52.120815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 831/1682] 10.222.0.23 () {54 vars in 921 bytes} [Fri Sep 9 16:31:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:31:52.146853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14d66b-fc8f-4596-a5ea-7df5f6bdab08 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:52.147400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14d66b-fc8f-4596-a5ea-7df5f6bdab08 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:52.147606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14d66b-fc8f-4596-a5ea-7df5f6bdab08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:52.191413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-7a14d66b-fc8f-4596-a5ea-7df5f6bdab08 None None] MFA Rules not processed for user `784b6147de91435292e4cf3940bcad9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:52.192940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7a14d66b-fc8f-4596-a5ea-7df5f6bdab08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:52.442886 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 852/1683] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Sep 9 16:31:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:31:52.477394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add397f9-3ff0-4912-b200-72a8eacf4b70 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:52.480432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-add397f9-3ff0-4912-b200-72a8eacf4b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:52.519146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add397f9-3ff0-4912-b200-72a8eacf4b70 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:52.538380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e36ca-45d0-438c-adb4-22f233e9eb5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784b6147de91435292e4cf3940bcad9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:52.539149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e36ca-45d0-438c-adb4-22f233e9eb5b service placement] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:52.539501 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e36ca-45d0-438c-adb4-22f233e9eb5b service placement] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:52.539836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e36ca-45d0-438c-adb4-22f233e9eb5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:52.565697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e36ca-45d0-438c-adb4-22f233e9eb5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:52.567605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e36ca-45d0-438c-adb4-22f233e9eb5b service placement] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:52.591296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 832/1684] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:31:52.625716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14d66b-fc8f-4596-a5ea-7df5f6bdab08 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:52.627937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7a14d66b-fc8f-4596-a5ea-7df5f6bdab08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:52.814960 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14d66b-fc8f-4596-a5ea-7df5f6bdab08 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:52.822317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff0875-a478-46af-88b5-6bbd77cfe1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:52.823622 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff0875-a478-46af-88b5-6bbd77cfe1ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:52.824262 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff0875-a478-46af-88b5-6bbd77cfe1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:52.824671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff0875-a478-46af-88b5-6bbd77cfe1ea admin admin] PATH_INFO: `/v3/users/00f532e0d03048cf86f9406d53e71d16` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:52.827060 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff0875-a478-46af-88b5-6bbd77cfe1ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=00f532e0d03048cf86f9406d53e71d16)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:52.829835 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff0875-a478-46af-88b5-6bbd77cfe1ea admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:52.934453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:52.954955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-03ff0875-a478-46af-88b5-6bbd77cfe1ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00f532e0d03048cf86f9406d53e71d16'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:52.955872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-03ff0875-a478-46af-88b5-6bbd77cfe1ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00f532e0d03048cf86f9406d53e71d16'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:52.958301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-03ff0875-a478-46af-88b5-6bbd77cfe1ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00f532e0d03048cf86f9406d53e71d16'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:52.971687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-03ff0875-a478-46af-88b5-6bbd77cfe1ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00f532e0d03048cf86f9406d53e71d16'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:52.998323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-03ff0875-a478-46af-88b5-6bbd77cfe1ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00f532e0d03048cf86f9406d53e71d16'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:53.032234 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 853/1685] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:31:52 2022] DELETE /identity/v3/users/00f532e0d03048cf86f9406d53e71d16 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:53.212440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519f2e2-377c-44f1-81ba-c14347a3f9dd None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:53.213900 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519f2e2-377c-44f1-81ba-c14347a3f9dd None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:53.214921 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519f2e2-377c-44f1-81ba-c14347a3f9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:53.298910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1519f2e2-377c-44f1-81ba-c14347a3f9dd None None] MFA Rules not processed for user `d095ed814b9e47e08d6b5a4fd2901c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:53.302083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1519f2e2-377c-44f1-81ba-c14347a3f9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:53.369875 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:53.399782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff0875-a478-46af-88b5-6bbd77cfe1ea admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:53.403584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-03ff0875-a478-46af-88b5-6bbd77cfe1ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:53.469670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 833/1686] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:53.470717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519f2e2-377c-44f1-81ba-c14347a3f9dd None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:53.472314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1519f2e2-377c-44f1-81ba-c14347a3f9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:53.633024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff0875-a478-46af-88b5-6bbd77cfe1ea admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:53.633300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519f2e2-377c-44f1-81ba-c14347a3f9dd None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:53.648837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:53.649561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:53.649842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:53.650116 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] PATH_INFO: `/v3/projects/cf44e7961d7e4ffcb7289c090e551583` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:53.651517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf44e7961d7e4ffcb7289c090e551583)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:53.652010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2d147-6e39-4d89-b118-80afccff3f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:53.652959 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2d147-6e39-4d89-b118-80afccff3f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:53.653431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2d147-6e39-4d89-b118-80afccff3f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:53.653595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:53.654040 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2d147-6e39-4d89-b118-80afccff3f55 admin admin] PATH_INFO: `/v3/projects/7a55eb91cec2492d96b7547af043e387` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:53.663048 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2d147-6e39-4d89-b118-80afccff3f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a55eb91cec2492d96b7547af043e387)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:53.665499 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2d147-6e39-4d89-b118-80afccff3f55 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:53.753200 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf44e7961d7e4ffcb7289c090e551583', 'request_id': 'req-53a4de37-6885-43ad-b227-6d53043e6c2f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:53.760606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-95e2d147-6e39-4d89-b118-80afccff3f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a55eb91cec2492d96b7547af043e387', 'request_id': 'req-95e2d147-6e39-4d89-b118-80afccff3f55'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:53.763783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf44e7961d7e4ffcb7289c090e551583', 'request_id': 'req-53a4de37-6885-43ad-b227-6d53043e6c2f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:53.775297 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-95e2d147-6e39-4d89-b118-80afccff3f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a55eb91cec2492d96b7547af043e387', 'request_id': 'req-95e2d147-6e39-4d89-b118-80afccff3f55'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:53.775492 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf44e7961d7e4ffcb7289c090e551583', 'request_id': 'req-53a4de37-6885-43ad-b227-6d53043e6c2f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:53.793593 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-95e2d147-6e39-4d89-b118-80afccff3f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a55eb91cec2492d96b7547af043e387', 'request_id': 'req-95e2d147-6e39-4d89-b118-80afccff3f55'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:53.808364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] The token cache is being invalidate because project cf44e7961d7e4ffcb7289c090e551583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:31:53.808675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:53.810807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:53.812428 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 834/1687] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:31:53 2022] DELETE /identity/v3/projects/cf44e7961d7e4ffcb7289c090e551583 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:53.813216 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:53.814267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-95e2d147-6e39-4d89-b118-80afccff3f55 admin admin] The token cache is being invalidate because project 7a55eb91cec2492d96b7547af043e387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:31:53.814825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-95e2d147-6e39-4d89-b118-80afccff3f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:53.816162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-95e2d147-6e39-4d89-b118-80afccff3f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:53.821025 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:53.826254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 854/1688] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:31:53 2022] DELETE /identity/v3/projects/7a55eb91cec2492d96b7547af043e387 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:53.827156 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20b1e1-a787-48cf-8eb7-1e918e4edef0 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:53.831137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20b1e1-a787-48cf-8eb7-1e918e4edef0 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:53.831590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20b1e1-a787-48cf-8eb7-1e918e4edef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:53.884938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-ac20b1e1-a787-48cf-8eb7-1e918e4edef0 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:53.889038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ac20b1e1-a787-48cf-8eb7-1e918e4edef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:53.958472 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:53.976990 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:54.009070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4de37-6885-43ad-b227-6d53043e6c2f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:54.055189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 855/1689] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:54.058962 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3139d1-cae1-4d65-b583-39afac2335f0 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:54.059738 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3139d1-cae1-4d65-b583-39afac2335f0 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:54.060595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3139d1-cae1-4d65-b583-39afac2335f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:54.138666 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-fc3139d1-cae1-4d65-b583-39afac2335f0 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:54.142090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-fc3139d1-cae1-4d65-b583-39afac2335f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:54.159073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed541d-9f2a-46f9-bb4d-4816315863dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:54.159711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed541d-9f2a-46f9-bb4d-4816315863dd service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:54.159990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed541d-9f2a-46f9-bb4d-4816315863dd service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:54.160313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed541d-9f2a-46f9-bb4d-4816315863dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:54.163860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2eed541d-9f2a-46f9-bb4d-4816315863dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:54.171814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 856/1690] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:54.173533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc80f17-69b0-41af-8462-1de9391d61c7 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:54.173916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc80f17-69b0-41af-8462-1de9391d61c7 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:54.174444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc80f17-69b0-41af-8462-1de9391d61c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:54.200167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-8dc80f17-69b0-41af-8462-1de9391d61c7 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:54.201536 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8dc80f17-69b0-41af-8462-1de9391d61c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:54.228013 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 857/1691] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:54.229291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc80f17-69b0-41af-8462-1de9391d61c7 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:54.230933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8dc80f17-69b0-41af-8462-1de9391d61c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:54.252181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc80f17-69b0-41af-8462-1de9391d61c7 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:54.268234 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:54.271016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:54.271358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:54.273532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] PATH_INFO: `/v3/projects/c2965cd2c2464e79b878aa09cb1d293d` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:54.275810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2965cd2c2464e79b878aa09cb1d293d)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:54.279355 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:54.297059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed541d-9f2a-46f9-bb4d-4816315863dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:54.298754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed541d-9f2a-46f9-bb4d-4816315863dd service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:54.315382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 835/1692] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:54.316934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a43fa-69e5-4966-8ede-2a9720f289db None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:54.317219 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a43fa-69e5-4966-8ede-2a9720f289db None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:54.317530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a43fa-69e5-4966-8ede-2a9720f289db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:54.363177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-452a43fa-69e5-4966-8ede-2a9720f289db None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:54.364737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-452a43fa-69e5-4966-8ede-2a9720f289db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:54.382904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 836/1693] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:54.383481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a43fa-69e5-4966-8ede-2a9720f289db None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:54.410527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a43fa-69e5-4966-8ede-2a9720f289db None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:54.423292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf0de91-82b7-4ad7-b751-1a528ab96a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:54.423963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0de91-82b7-4ad7-b751-1a528ab96a3b admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:54.424243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0de91-82b7-4ad7-b751-1a528ab96a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:54.424518 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0de91-82b7-4ad7-b751-1a528ab96a3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:54.425624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf0de91-82b7-4ad7-b751-1a528ab96a3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:31:54.426052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf0de91-82b7-4ad7-b751-1a528ab96a3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:54.427765 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf0de91-82b7-4ad7-b751-1a528ab96a3b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:54.456836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 837/1694] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:31:54.457883 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf0de91-82b7-4ad7-b751-1a528ab96a3b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:54.491992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf0de91-82b7-4ad7-b751-1a528ab96a3b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:54.525348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2965cd2c2464e79b878aa09cb1d293d', 'request_id': 'req-1442e870-884c-4ae7-9af4-7a33e6e3a334'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:54.569678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2965cd2c2464e79b878aa09cb1d293d', 'request_id': 'req-1442e870-884c-4ae7-9af4-7a33e6e3a334'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:54.602108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2965cd2c2464e79b878aa09cb1d293d', 'request_id': 'req-1442e870-884c-4ae7-9af4-7a33e6e3a334'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:54.675798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] The token cache is being invalidate because project c2965cd2c2464e79b878aa09cb1d293d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:31:54.680747 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:54.681988 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:54.684838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 858/1695] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:31:54 2022] DELETE /identity/v3/projects/c2965cd2c2464e79b878aa09cb1d293d => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:54.687196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:54.692997 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:54.700886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe612f7-0626-4972-acba-860e381ace2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:54.701635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe612f7-0626-4972-acba-860e381ace2d admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:54.701809 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe612f7-0626-4972-acba-860e381ace2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:54.702085 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe612f7-0626-4972-acba-860e381ace2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:54.713550 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe612f7-0626-4972-acba-860e381ace2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:31:54.718056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe612f7-0626-4972-acba-860e381ace2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:54.721673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe612f7-0626-4972-acba-860e381ace2d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:54.734820 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442e870-884c-4ae7-9af4-7a33e6e3a334 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:54.747857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e17dd-fe77-47f0-bf20-bc1f32b6ded9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:54.748715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e17dd-fe77-47f0-bf20-bc1f32b6ded9 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:54.749169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e17dd-fe77-47f0-bf20-bc1f32b6ded9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:54.749714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e17dd-fe77-47f0-bf20-bc1f32b6ded9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:54.750545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e17dd-fe77-47f0-bf20-bc1f32b6ded9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:54.753815 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e17dd-fe77-47f0-bf20-bc1f32b6ded9 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:54.805069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 838/1696] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:31:54.808364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe612f7-0626-4972-acba-860e381ace2d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:54.817136 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cbe612f7-0626-4972-acba-860e381ace2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:54.856182 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 859/1697] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:54 2022] POST /identity/v3/projects => generated 389 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:54.859514 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e17dd-fe77-47f0-bf20-bc1f32b6ded9 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:54.878828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e51e17dd-fe77-47f0-bf20-bc1f32b6ded9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:54.887191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe612f7-0626-4972-acba-860e381ace2d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:54.909165 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:55.102759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e17dd-fe77-47f0-bf20-bc1f32b6ded9 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:55.102997 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc5876-ee14-41f7-9f7d-f86023a59f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:55.105022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc5876-ee14-41f7-9f7d-f86023a59f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:55.105444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc5876-ee14-41f7-9f7d-f86023a59f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:55.105717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc5876-ee14-41f7-9f7d-f86023a59f50 admin admin] PATH_INFO: `/v3/users/46e3bdbed41d44c8a2362b83fb6d25c9` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:55.116646 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc5876-ee14-41f7-9f7d-f86023a59f50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46e3bdbed41d44c8a2362b83fb6d25c9)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:55.124217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c277475-86e2-4434-baae-a467ecc7e036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:55.126015 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c277475-86e2-4434-baae-a467ecc7e036 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:55.126181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc5876-ee14-41f7-9f7d-f86023a59f50 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:55.127140 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c277475-86e2-4434-baae-a467ecc7e036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:55.127497 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c277475-86e2-4434-baae-a467ecc7e036 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:55.129257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c277475-86e2-4434-baae-a467ecc7e036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:55.135085 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c277475-86e2-4434-baae-a467ecc7e036 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:55.217834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 860/1698] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:54 2022] POST /identity/v3/projects => generated 385 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:55.217834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c277475-86e2-4434-baae-a467ecc7e036 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:55.240926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:55.250529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3bcc5876-ee14-41f7-9f7d-f86023a59f50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46e3bdbed41d44c8a2362b83fb6d25c9'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:55.252500 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3bcc5876-ee14-41f7-9f7d-f86023a59f50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46e3bdbed41d44c8a2362b83fb6d25c9'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:55.253732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3bcc5876-ee14-41f7-9f7d-f86023a59f50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46e3bdbed41d44c8a2362b83fb6d25c9'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:55.284882 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3bcc5876-ee14-41f7-9f7d-f86023a59f50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46e3bdbed41d44c8a2362b83fb6d25c9'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:55.318803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3bcc5876-ee14-41f7-9f7d-f86023a59f50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46e3bdbed41d44c8a2362b83fb6d25c9'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:55.347033 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c277475-86e2-4434-baae-a467ecc7e036 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:55.350551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 839/1699] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:31:54 2022] DELETE /identity/v3/users/46e3bdbed41d44c8a2362b83fb6d25c9 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:55.356369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc5876-ee14-41f7-9f7d-f86023a59f50 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:55.358346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3bcc5876-ee14-41f7-9f7d-f86023a59f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:55.358548 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de96e1-1627-444b-bf1a-31d003256b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:55.359200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de96e1-1627-444b-bf1a-31d003256b42 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:55.359479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de96e1-1627-444b-bf1a-31d003256b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:55.359749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de96e1-1627-444b-bf1a-31d003256b42 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:55.360391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de96e1-1627-444b-bf1a-31d003256b42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:55.362396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de96e1-1627-444b-bf1a-31d003256b42 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:55.413969 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 861/1700] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:55 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:55.413969 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cae62-b3a6-45c9-8645-0832606e0521 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:55.413969 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cae62-b3a6-45c9-8645-0832606e0521 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:55.413969 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cae62-b3a6-45c9-8645-0832606e0521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:55.446084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc5876-ee14-41f7-9f7d-f86023a59f50 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:55.452118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d2e7e-d573-4da8-bbbd-8aedf1b997d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:55.452952 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d2e7e-d573-4da8-bbbd-8aedf1b997d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:55.453735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d2e7e-d573-4da8-bbbd-8aedf1b997d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:55.454179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d2e7e-d573-4da8-bbbd-8aedf1b997d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:55.454993 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d2e7e-d573-4da8-bbbd-8aedf1b997d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:55.457191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d2e7e-d573-4da8-bbbd-8aedf1b997d6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:55.481396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-de4cae62-b3a6-45c9-8645-0832606e0521 None None] MFA Rules not processed for user `1ce1cd9c19584549b794df8e487abf00`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:55.482806 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-de4cae62-b3a6-45c9-8645-0832606e0521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:55.491976 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 840/1701] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:55 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:55.494114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d2e7e-d573-4da8-bbbd-8aedf1b997d6 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:55.496280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bc0d2e7e-d573-4da8-bbbd-8aedf1b997d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:55.512931 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d2e7e-d573-4da8-bbbd-8aedf1b997d6 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:55.518509 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ea982-45f8-4026-8eca-f81987d8ee4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:55.519160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ea982-45f8-4026-8eca-f81987d8ee4a admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:55.519443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ea982-45f8-4026-8eca-f81987d8ee4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:55.519809 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ea982-45f8-4026-8eca-f81987d8ee4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:55.521022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ea982-45f8-4026-8eca-f81987d8ee4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:55.523051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ea982-45f8-4026-8eca-f81987d8ee4a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:55.540713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 841/1702] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:55.542104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ea982-45f8-4026-8eca-f81987d8ee4a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:55.550014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:55.560588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ea982-45f8-4026-8eca-f81987d8ee4a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:55.567668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f6ec1-9b42-41f5-ae7b-2ae8fed4b223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:55.568796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f6ec1-9b42-41f5-ae7b-2ae8fed4b223 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:55.570489 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f6ec1-9b42-41f5-ae7b-2ae8fed4b223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:55.570702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f6ec1-9b42-41f5-ae7b-2ae8fed4b223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:55.571371 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f6ec1-9b42-41f5-ae7b-2ae8fed4b223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:55.573274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f6ec1-9b42-41f5-ae7b-2ae8fed4b223 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:55.589049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 842/1703] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:55.593459 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f6ec1-9b42-41f5-ae7b-2ae8fed4b223 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:55.606936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f6ec1-9b42-41f5-ae7b-2ae8fed4b223 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:55.618091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f3068-5350-40be-b4ae-056f3923c558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:55.618091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f3068-5350-40be-b4ae-056f3923c558 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:55.618091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f3068-5350-40be-b4ae-056f3923c558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:55.618091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f3068-5350-40be-b4ae-056f3923c558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:55.618580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f3068-5350-40be-b4ae-056f3923c558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:55.618580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f3068-5350-40be-b4ae-056f3923c558 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:55.626955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 862/1704] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:55.627925 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cae62-b3a6-45c9-8645-0832606e0521 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:55.637924 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 843/1705] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:55.639787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613cc04-e4b2-4b5e-bab3-c4cf24fdff98 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:55.640184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613cc04-e4b2-4b5e-bab3-c4cf24fdff98 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:55.640569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613cc04-e4b2-4b5e-bab3-c4cf24fdff98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:55.657046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cae62-b3a6-45c9-8645-0832606e0521 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:55.662136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9811df-a46b-405c-8be4-3bd0a654bec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:55.662812 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9811df-a46b-405c-8be4-3bd0a654bec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:55.663093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9811df-a46b-405c-8be4-3bd0a654bec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:55.663364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9811df-a46b-405c-8be4-3bd0a654bec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:55.664035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9811df-a46b-405c-8be4-3bd0a654bec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:55.665820 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9811df-a46b-405c-8be4-3bd0a654bec3 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:55.667943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-9613cc04-e4b2-4b5e-bab3-c4cf24fdff98 None None] MFA Rules not processed for user `1ce1cd9c19584549b794df8e487abf00`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:55.670464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9613cc04-e4b2-4b5e-bab3-c4cf24fdff98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:55.680236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 844/1706] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:55.681162 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613cc04-e4b2-4b5e-bab3-c4cf24fdff98 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:55.690059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 863/1707] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:55.691689 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9811df-a46b-405c-8be4-3bd0a654bec3 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:55.697594 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7a9811df-a46b-405c-8be4-3bd0a654bec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:55.724516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613cc04-e4b2-4b5e-bab3-c4cf24fdff98 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:55.752522 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bbd65-630d-4662-87ca-f58f79a95fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:55.752522 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bbd65-630d-4662-87ca-f58f79a95fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:55.752522 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bbd65-630d-4662-87ca-f58f79a95fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:55.752904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bbd65-630d-4662-87ca-f58f79a95fc2 admin admin] PATH_INFO: `/v3/projects/c803181092054dc1ab3a60554871c67a/users/28111b0c295541b78cf67ba86035c36c/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:55.819526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76bbd65-630d-4662-87ca-f58f79a95fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c803181092054dc1ab3a60554871c67a, user_id=28111b0c295541b78cf67ba86035c36c, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:55.824339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76bbd65-630d-4662-87ca-f58f79a95fc2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:55.885352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 845/1708] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:55 2022] PUT /identity/v3/projects/c803181092054dc1ab3a60554871c67a/users/28111b0c295541b78cf67ba86035c36c/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:55.886517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bbd65-630d-4662-87ca-f58f79a95fc2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:55.892156 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9811df-a46b-405c-8be4-3bd0a654bec3 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:55.928281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:56.062538 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bbd65-630d-4662-87ca-f58f79a95fc2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:56.087650 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c1b56-8a9a-45d1-8afe-e6f3428fa20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:56.088508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c1b56-8a9a-45d1-8afe-e6f3428fa20e admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:56.088795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c1b56-8a9a-45d1-8afe-e6f3428fa20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:56.089162 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c1b56-8a9a-45d1-8afe-e6f3428fa20e admin admin] PATH_INFO: `/v3/projects/d4daa77b96154ac480e09962d6035b49/users/250dcdef3e234a438e0d0aaa6fbdfaab/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:56.125769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44502281-d0d1-42d3-9e41-4bf001d4cdcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:56.126540 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c1b56-8a9a-45d1-8afe-e6f3428fa20e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4daa77b96154ac480e09962d6035b49, user_id=250dcdef3e234a438e0d0aaa6fbdfaab, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:56.127870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44502281-d0d1-42d3-9e41-4bf001d4cdcb service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:56.129690 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c1b56-8a9a-45d1-8afe-e6f3428fa20e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:56.130159 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44502281-d0d1-42d3-9e41-4bf001d4cdcb service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:56.132828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44502281-d0d1-42d3-9e41-4bf001d4cdcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:56.162991 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 846/1709] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:55 2022] PUT /identity/v3/projects/d4daa77b96154ac480e09962d6035b49/users/250dcdef3e234a438e0d0aaa6fbdfaab/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:56.164291 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b186ed-7d90-48b4-afe4-412ff127f281 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:56.164574 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b186ed-7d90-48b4-afe4-412ff127f281 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:56.164845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b186ed-7d90-48b4-afe4-412ff127f281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:56.224129 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-97b186ed-7d90-48b4-afe4-412ff127f281 None None] MFA Rules not processed for user `53c0721dc18343b18993ff5af098841d`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:56.225367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-97b186ed-7d90-48b4-afe4-412ff127f281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:56.351568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44502281-d0d1-42d3-9e41-4bf001d4cdcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:56.353536 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44502281-d0d1-42d3-9e41-4bf001d4cdcb service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:56.374227 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 864/1710] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:56.386339 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87caff9-557f-45ad-9d77-b4a750e9d0a2 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:56.386879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87caff9-557f-45ad-9d77-b4a750e9d0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:56.387472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87caff9-557f-45ad-9d77-b4a750e9d0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:56.389256 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 847/1711] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:56.390754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146e0b9-2ad7-4040-9097-0be7d1df1e19 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:56.391046 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146e0b9-2ad7-4040-9097-0be7d1df1e19 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:56.391443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146e0b9-2ad7-4040-9097-0be7d1df1e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:56.453704 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8146e0b9-2ad7-4040-9097-0be7d1df1e19 None None] MFA Rules not processed for user `250dcdef3e234a438e0d0aaa6fbdfaab`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:56.454684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8146e0b9-2ad7-4040-9097-0be7d1df1e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:56.469587 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-c87caff9-557f-45ad-9d77-b4a750e9d0a2 None None] MFA Rules not processed for user `28111b0c295541b78cf67ba86035c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:56.470994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c87caff9-557f-45ad-9d77-b4a750e9d0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:56.481463 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:56.623321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 848/1712] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:56.624200 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146e0b9-2ad7-4040-9097-0be7d1df1e19 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:56.628551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8146e0b9-2ad7-4040-9097-0be7d1df1e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:56.691745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 865/1713] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:56.692915 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87caff9-557f-45ad-9d77-b4a750e9d0a2 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:56.696501 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c87caff9-557f-45ad-9d77-b4a750e9d0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:56.834252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146e0b9-2ad7-4040-9097-0be7d1df1e19 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:56.843938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:56.856703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11338522-5ad6-4916-aaed-244e70e1b51f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:56.857350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11338522-5ad6-4916-aaed-244e70e1b51f service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:56.857627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11338522-5ad6-4916-aaed-244e70e1b51f service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:56.857893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11338522-5ad6-4916-aaed-244e70e1b51f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:56.876506 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87caff9-557f-45ad-9d77-b4a750e9d0a2 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:56.888233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:56.888417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11338522-5ad6-4916-aaed-244e70e1b51f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:56.889055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:56.889421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:56.889831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] PATH_INFO: `/v3/projects/7ada5ae060964366b003f3a329d4b066` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:56.890047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11338522-5ad6-4916-aaed-244e70e1b51f service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:56.891747 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ada5ae060964366b003f3a329d4b066)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:56.893824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:56.908126 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 849/1714] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:56.909041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11338522-5ad6-4916-aaed-244e70e1b51f service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:56.958186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11338522-5ad6-4916-aaed-244e70e1b51f service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:56.966471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2df051-f4f2-4f97-8577-0e458cf0e92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:56.967195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2df051-f4f2-4f97-8577-0e458cf0e92f admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:56.967481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2df051-f4f2-4f97-8577-0e458cf0e92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:56.967747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2df051-f4f2-4f97-8577-0e458cf0e92f admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:56.968394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2df051-f4f2-4f97-8577-0e458cf0e92f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:56.970337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2df051-f4f2-4f97-8577-0e458cf0e92f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:57.038659 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 850/1715] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:56 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:57.044868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2df051-f4f2-4f97-8577-0e458cf0e92f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:57.100567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ada5ae060964366b003f3a329d4b066', 'request_id': 'req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:57.115319 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ada5ae060964366b003f3a329d4b066', 'request_id': 'req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:57.133977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ada5ae060964366b003f3a329d4b066', 'request_id': 'req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:57.148128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] The token cache is being invalidate because project 7ada5ae060964366b003f3a329d4b066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:31:57.148652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:57.149564 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:31:57.153772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 866/1716] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:31:56 2022] DELETE /identity/v3/projects/7ada5ae060964366b003f3a329d4b066 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:57.154942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:57.160664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:57.262120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2df051-f4f2-4f97-8577-0e458cf0e92f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:57.302134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:57.315869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0b26e-9a67-4f6f-95ef-d1acc7283d12 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:57.338703 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81533331-5a10-4ce0-8b00-115b73c301b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:57.339367 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81533331-5a10-4ce0-8b00-115b73c301b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:57.339641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81533331-5a10-4ce0-8b00-115b73c301b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:57.339983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81533331-5a10-4ce0-8b00-115b73c301b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:57.340645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81533331-5a10-4ce0-8b00-115b73c301b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:57.342843 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81533331-5a10-4ce0-8b00-115b73c301b0 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:57.404401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 867/1717] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:57 2022] POST /identity/v3/users => generated 299 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:57.410819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69bac2-0d86-48c7-be8a-d5a16c9b38ad None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:57.411174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69bac2-0d86-48c7-be8a-d5a16c9b38ad None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:57.411453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69bac2-0d86-48c7-be8a-d5a16c9b38ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:57.480783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8d119-ec3b-42b6-bbb9-84e6efd492bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:57.482836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8d119-ec3b-42b6-bbb9-84e6efd492bd admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:57.484952 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8d119-ec3b-42b6-bbb9-84e6efd492bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:57.485436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8d119-ec3b-42b6-bbb9-84e6efd492bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:57.485566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-9a69bac2-0d86-48c7-be8a-d5a16c9b38ad None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:57.486620 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8d119-ec3b-42b6-bbb9-84e6efd492bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:57.488042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-9a69bac2-0d86-48c7-be8a-d5a16c9b38ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:57.489913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8d119-ec3b-42b6-bbb9-84e6efd492bd admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:57.523542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 868/1718] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:57.524428 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69bac2-0d86-48c7-be8a-d5a16c9b38ad None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:57.538291 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 851/1719] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:57 2022] POST /identity/v3/projects => generated 387 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:57.540486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff43fb8-00a0-4956-aece-d2ea7bd1ff01 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:57.540784 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff43fb8-00a0-4956-aece-d2ea7bd1ff01 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:57.541802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff43fb8-00a0-4956-aece-d2ea7bd1ff01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:57.570859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69bac2-0d86-48c7-be8a-d5a16c9b38ad None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:57.611866 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4ff43fb8-00a0-4956-aece-d2ea7bd1ff01 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:57.613589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4ff43fb8-00a0-4956-aece-d2ea7bd1ff01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:57.711460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:57.725602 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a839eaa-c89b-4f30-b7d8-91d42253e520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:57.725980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a839eaa-c89b-4f30-b7d8-91d42253e520 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:57.727320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a839eaa-c89b-4f30-b7d8-91d42253e520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:57.727511 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a839eaa-c89b-4f30-b7d8-91d42253e520 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:57.728183 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a839eaa-c89b-4f30-b7d8-91d42253e520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:57.731876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a839eaa-c89b-4f30-b7d8-91d42253e520 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:57.753893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 869/1720] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:57.757317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a839eaa-c89b-4f30-b7d8-91d42253e520 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:57.759213 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5a839eaa-c89b-4f30-b7d8-91d42253e520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:57.782256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a839eaa-c89b-4f30-b7d8-91d42253e520 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:57.795044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 852/1721] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:57.795044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff43fb8-00a0-4956-aece-d2ea7bd1ff01 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:57.800376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be13af-77b3-4893-b58f-2b0801c0df7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:57.801182 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be13af-77b3-4893-b58f-2b0801c0df7b admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:57.801385 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be13af-77b3-4893-b58f-2b0801c0df7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:57.801583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be13af-77b3-4893-b58f-2b0801c0df7b admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:57.802230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be13af-77b3-4893-b58f-2b0801c0df7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:57.804224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be13af-77b3-4893-b58f-2b0801c0df7b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:57.813733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff43fb8-00a0-4956-aece-d2ea7bd1ff01 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:57.833669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d53ce-2f2e-4853-9f76-b076964c54d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:57.834433 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d53ce-2f2e-4853-9f76-b076964c54d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:57.834738 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d53ce-2f2e-4853-9f76-b076964c54d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:57.835116 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d53ce-2f2e-4853-9f76-b076964c54d3 admin admin] PATH_INFO: `/v3/projects/563ee41fae234711992a8d9289a9984d/users/14362fb1f81f4cf29256e35caec11afa/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:57.852252 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 870/1722] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:57 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:57.853375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be13af-77b3-4893-b58f-2b0801c0df7b admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:57.882414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be13af-77b3-4893-b58f-2b0801c0df7b admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:57.888788 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d53ce-2f2e-4853-9f76-b076964c54d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=563ee41fae234711992a8d9289a9984d, user_id=14362fb1f81f4cf29256e35caec11afa, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:57.898234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d53ce-2f2e-4853-9f76-b076964c54d3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:57.909025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182c4f8-a065-4f32-b89c-41475222a9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:57.909836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182c4f8-a065-4f32-b89c-41475222a9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:57.910162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182c4f8-a065-4f32-b89c-41475222a9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:57.910425 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182c4f8-a065-4f32-b89c-41475222a9a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:57.911211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182c4f8-a065-4f32-b89c-41475222a9a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:31:57.911451 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182c4f8-a065-4f32-b89c-41475222a9a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:57.913141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182c4f8-a065-4f32-b89c-41475222a9a2 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:57.935208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 853/1723] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:57 2022] PUT /identity/v3/projects/563ee41fae234711992a8d9289a9984d/users/14362fb1f81f4cf29256e35caec11afa/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:57.936145 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d53ce-2f2e-4853-9f76-b076964c54d3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:57.961720 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 871/1724] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:31:57.962923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc893bc7-bd5c-4992-bd25-da11c251c2a6 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:57.963201 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc893bc7-bd5c-4992-bd25-da11c251c2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:57.963473 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc893bc7-bd5c-4992-bd25-da11c251c2a6 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:57.973336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 872/1725] 10.222.0.23 () {54 vars in 912 bytes} [Fri Sep 9 16:31:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:31:57.974424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc893bc7-bd5c-4992-bd25-da11c251c2a6 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:58.120929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d53ce-2f2e-4853-9f76-b076964c54d3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:58.140529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8e21c-cd37-433c-8458-2707670e10f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:58.141078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8e21c-cd37-433c-8458-2707670e10f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:58.141353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8e21c-cd37-433c-8458-2707670e10f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:58.141628 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8e21c-cd37-433c-8458-2707670e10f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:58.142323 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8e21c-cd37-433c-8458-2707670e10f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:58.143729 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc893bc7-bd5c-4992-bd25-da11c251c2a6 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:58.143971 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8e21c-cd37-433c-8458-2707670e10f6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:58.155353 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603fa9f-9872-4864-b821-ddd7325f49f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:58.156151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603fa9f-9872-4864-b821-ddd7325f49f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:58.156283 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603fa9f-9872-4864-b821-ddd7325f49f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:58.156497 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603fa9f-9872-4864-b821-ddd7325f49f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:58.157155 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603fa9f-9872-4864-b821-ddd7325f49f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:58.161204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 854/1726] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:58.164736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8e21c-cd37-433c-8458-2707670e10f6 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:58.166482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603fa9f-9872-4864-b821-ddd7325f49f5 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:58.196898 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 873/1727] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:58.196898 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e04e4-8c91-46f2-a63e-708df3ff117c None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:58.197632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e04e4-8c91-46f2-a63e-708df3ff117c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:58.197632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e04e4-8c91-46f2-a63e-708df3ff117c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:58.200772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8e21c-cd37-433c-8458-2707670e10f6 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:58.211528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dcf6d-bf43-47d1-8b1f-d21461a8c68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:58.212304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dcf6d-bf43-47d1-8b1f-d21461a8c68b admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:58.212691 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dcf6d-bf43-47d1-8b1f-d21461a8c68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:58.213786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dcf6d-bf43-47d1-8b1f-d21461a8c68b admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:58.214598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7dcf6d-bf43-47d1-8b1f-d21461a8c68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:58.216974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7dcf6d-bf43-47d1-8b1f-d21461a8c68b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:58.274091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:58.286153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 855/1728] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:31:58 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:58.286153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dcf6d-bf43-47d1-8b1f-d21461a8c68b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:58.298071 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-a53e04e4-8c91-46f2-a63e-708df3ff117c None None] MFA Rules not processed for user `aa65b6854dc54e289cc3e8e0b3746dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:58.299411 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a53e04e4-8c91-46f2-a63e-708df3ff117c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:58.402489 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dcf6d-bf43-47d1-8b1f-d21461a8c68b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:58.414922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50467f48-07c6-4c0b-93bc-43beda51122f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:58.415523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50467f48-07c6-4c0b-93bc-43beda51122f admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:58.415799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50467f48-07c6-4c0b-93bc-43beda51122f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:58.416068 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50467f48-07c6-4c0b-93bc-43beda51122f admin admin] PATH_INFO: `/v3/projects/8ed6fc8b91ff468895970f2d99e84b70/users/9217b3d8de9449208e1b9298a30ca009/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:58.427031 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 874/1729] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:31:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:58.427912 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e04e4-8c91-46f2-a63e-708df3ff117c None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:58.450726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e04e4-8c91-46f2-a63e-708df3ff117c None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:58.451299 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50467f48-07c6-4c0b-93bc-43beda51122f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed6fc8b91ff468895970f2d99e84b70, user_id=9217b3d8de9449208e1b9298a30ca009, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:58.457038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0d44b-820c-400b-bc95-bec3da943bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:58.457755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d44b-820c-400b-bc95-bec3da943bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:58.458291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d44b-820c-400b-bc95-bec3da943bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:58.458364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d44b-820c-400b-bc95-bec3da943bf5 admin admin] PATH_INFO: `/v3/projects/563ee41fae234711992a8d9289a9984d/users/14362fb1f81f4cf29256e35caec11afa/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:58.460142 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50467f48-07c6-4c0b-93bc-43beda51122f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:58.462317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0d44b-820c-400b-bc95-bec3da943bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=563ee41fae234711992a8d9289a9984d, user_id=14362fb1f81f4cf29256e35caec11afa, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:58.464518 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0d44b-820c-400b-bc95-bec3da943bf5 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:58.476658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 875/1730] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:58 2022] PUT /identity/v3/projects/563ee41fae234711992a8d9289a9984d/users/14362fb1f81f4cf29256e35caec11afa/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:58.478263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785384e8-0d74-49e0-a143-fd6e3ae10de5 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:58.478263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785384e8-0d74-49e0-a143-fd6e3ae10de5 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:58.478546 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785384e8-0d74-49e0-a143-fd6e3ae10de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:58.490821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 856/1731] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:58 2022] PUT /identity/v3/projects/8ed6fc8b91ff468895970f2d99e84b70/users/9217b3d8de9449208e1b9298a30ca009/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:58.491910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50467f48-07c6-4c0b-93bc-43beda51122f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:58.536324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-785384e8-0d74-49e0-a143-fd6e3ae10de5 None None] MFA Rules not processed for user `67d86f09c6304121be2021cb5c15e63e`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:58.540158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-785384e8-0d74-49e0-a143-fd6e3ae10de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:58.582259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50467f48-07c6-4c0b-93bc-43beda51122f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:58.588691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:58.592301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823935b-1d54-46c8-802c-b7ebe361600b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:58.594097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823935b-1d54-46c8-802c-b7ebe361600b admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:58.594807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823935b-1d54-46c8-802c-b7ebe361600b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:58.595179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823935b-1d54-46c8-802c-b7ebe361600b admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:58.596311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823935b-1d54-46c8-802c-b7ebe361600b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:58.603710 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823935b-1d54-46c8-802c-b7ebe361600b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:58.645191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 857/1732] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:31:58 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:31:58.646555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823935b-1d54-46c8-802c-b7ebe361600b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:58.650738 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c823935b-1d54-46c8-802c-b7ebe361600b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:58.661171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 876/1733] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:58.661171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bded86-1be2-4460-bb2f-94d956df4b7a None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:58.661171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bded86-1be2-4460-bb2f-94d956df4b7a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:58.661171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bded86-1be2-4460-bb2f-94d956df4b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:58.725602 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d3bded86-1be2-4460-bb2f-94d956df4b7a None None] MFA Rules not processed for user `14362fb1f81f4cf29256e35caec11afa`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:58.729294 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d3bded86-1be2-4460-bb2f-94d956df4b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:58.786276 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823935b-1d54-46c8-802c-b7ebe361600b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:58.794635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c1330-fa0e-4f67-9797-e25d295b1f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:58.795412 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c1330-fa0e-4f67-9797-e25d295b1f99 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:58.795791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c1330-fa0e-4f67-9797-e25d295b1f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:58.801159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c1330-fa0e-4f67-9797-e25d295b1f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:58.820225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c1330-fa0e-4f67-9797-e25d295b1f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:58.823507 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c1330-fa0e-4f67-9797-e25d295b1f99 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:58.853727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 858/1734] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:58.855054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c1330-fa0e-4f67-9797-e25d295b1f99 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:58.870823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c1330-fa0e-4f67-9797-e25d295b1f99 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:58.871934 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 877/1735] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:58.873302 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bded86-1be2-4460-bb2f-94d956df4b7a None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:58.877208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23737ef-aa45-4af3-80d5-8518e8d9a43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:58.877823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23737ef-aa45-4af3-80d5-8518e8d9a43e admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:58.878158 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23737ef-aa45-4af3-80d5-8518e8d9a43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:58.878431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23737ef-aa45-4af3-80d5-8518e8d9a43e admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:58.879547 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23737ef-aa45-4af3-80d5-8518e8d9a43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:58.881354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23737ef-aa45-4af3-80d5-8518e8d9a43e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:58.889963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bded86-1be2-4460-bb2f-94d956df4b7a None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:58.899121 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 859/1736] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:58.899426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc313e7-1a19-48ef-8b75-1605c075d9a4 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:58.899644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc313e7-1a19-48ef-8b75-1605c075d9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:58.899981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc313e7-1a19-48ef-8b75-1605c075d9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:58.902862 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d94f4-14ba-4897-ba09-dea91cf94412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:58.903781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d94f4-14ba-4897-ba09-dea91cf94412 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:58.906466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d94f4-14ba-4897-ba09-dea91cf94412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:58.907010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d94f4-14ba-4897-ba09-dea91cf94412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:58.909211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d94f4-14ba-4897-ba09-dea91cf94412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:58.909988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:58.914129 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d94f4-14ba-4897-ba09-dea91cf94412 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:58.927370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7bc313e7-1a19-48ef-8b75-1605c075d9a4 None None] MFA Rules not processed for user `67d86f09c6304121be2021cb5c15e63e`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:58.928995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7bc313e7-1a19-48ef-8b75-1605c075d9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:58.946209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 860/1738] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:58.946415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 878/1738] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:58.946415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d94f4-14ba-4897-ba09-dea91cf94412 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:58.946631 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc313e7-1a19-48ef-8b75-1605c075d9a4 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:58.955611 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7bc313e7-1a19-48ef-8b75-1605c075d9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:58.957847 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d94f4-14ba-4897-ba09-dea91cf94412 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:58.982658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a89af7-5299-4500-8f19-262a901618cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:58.983366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a89af7-5299-4500-8f19-262a901618cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:58.983658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a89af7-5299-4500-8f19-262a901618cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:58.983936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a89af7-5299-4500-8f19-262a901618cc admin admin] PATH_INFO: `/v3/projects/8ed6fc8b91ff468895970f2d99e84b70/users/9217b3d8de9449208e1b9298a30ca009/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:58.987307 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a89af7-5299-4500-8f19-262a901618cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed6fc8b91ff468895970f2d99e84b70, user_id=9217b3d8de9449208e1b9298a30ca009, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:58.990074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a89af7-5299-4500-8f19-262a901618cc admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:59.004895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 879/1739] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:58 2022] PUT /identity/v3/projects/8ed6fc8b91ff468895970f2d99e84b70/users/9217b3d8de9449208e1b9298a30ca009/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:59.005908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a89af7-5299-4500-8f19-262a901618cc admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:59.029801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc313e7-1a19-48ef-8b75-1605c075d9a4 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:59.097269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a89af7-5299-4500-8f19-262a901618cc admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:59.104489 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a20037-d1a6-4cc6-aeca-12b5af4f5c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:59.105885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a20037-d1a6-4cc6-aeca-12b5af4f5c64 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:59.106284 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a20037-d1a6-4cc6-aeca-12b5af4f5c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:59.106721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a20037-d1a6-4cc6-aeca-12b5af4f5c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:59.107861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a20037-d1a6-4cc6-aeca-12b5af4f5c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:59.109529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a372a-dc15-4b9c-a4f0-66a7ebc1fcd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:59.110154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a372a-dc15-4b9c-a4f0-66a7ebc1fcd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:59.110370 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a20037-d1a6-4cc6-aeca-12b5af4f5c64 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:59.110742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a372a-dc15-4b9c-a4f0-66a7ebc1fcd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:59.111014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a372a-dc15-4b9c-a4f0-66a7ebc1fcd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:59.123790 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 880/1740] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:31:59.125735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a20037-d1a6-4cc6-aeca-12b5af4f5c64 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:59.128781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-13a20037-d1a6-4cc6-aeca-12b5af4f5c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:59.190737 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:59.225807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a372a-dc15-4b9c-a4f0-66a7ebc1fcd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:59.227641 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a372a-dc15-4b9c-a4f0-66a7ebc1fcd2 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:59.243337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 861/1741] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:59.245020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717d456-8119-4a72-b6dd-8ecf377faaff None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:59.245573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717d456-8119-4a72-b6dd-8ecf377faaff None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:59.246126 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717d456-8119-4a72-b6dd-8ecf377faaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:59.252956 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a20037-d1a6-4cc6-aeca-12b5af4f5c64 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:59.277530 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f72dc8-bc2d-4398-b350-3cfd656b86f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:59.278202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f72dc8-bc2d-4398-b350-3cfd656b86f4 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:59.289617 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f72dc8-bc2d-4398-b350-3cfd656b86f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:59.290744 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f72dc8-bc2d-4398-b350-3cfd656b86f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:59.299125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d717d456-8119-4a72-b6dd-8ecf377faaff None None] MFA Rules not processed for user `9217b3d8de9449208e1b9298a30ca009`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:59.301348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d717d456-8119-4a72-b6dd-8ecf377faaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:59.368404 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:31:59.392410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 862/1742] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:59 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:59.393685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717d456-8119-4a72-b6dd-8ecf377faaff None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:59.405603 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717d456-8119-4a72-b6dd-8ecf377faaff None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:59.411246 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61eb0fc-1505-43cc-a12b-bf2406586f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:59.412072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eb0fc-1505-43cc-a12b-bf2406586f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:59.412463 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eb0fc-1505-43cc-a12b-bf2406586f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:59.412843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eb0fc-1505-43cc-a12b-bf2406586f0a admin admin] PATH_INFO: `/v3/projects/940ba6cb0f2f47b98a21a9816e16d2d9/users/36e9c4d95afa44488f82a1110c8571f7/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:59.423907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f72dc8-bc2d-4398-b350-3cfd656b86f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:59.426297 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f72dc8-bc2d-4398-b350-3cfd656b86f4 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:59.432994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61eb0fc-1505-43cc-a12b-bf2406586f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=940ba6cb0f2f47b98a21a9816e16d2d9, user_id=36e9c4d95afa44488f82a1110c8571f7, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:59.436920 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61eb0fc-1505-43cc-a12b-bf2406586f0a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:59.450976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 881/1743] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:59.452613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e2a03-0a08-465a-8051-acb3d8d6ec58 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:59.454116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e2a03-0a08-465a-8051-acb3d8d6ec58 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:59.455784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e2a03-0a08-465a-8051-acb3d8d6ec58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:59.459033 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 863/1744] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:31:59 2022] PUT /identity/v3/projects/940ba6cb0f2f47b98a21a9816e16d2d9/users/36e9c4d95afa44488f82a1110c8571f7/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:31:59.460757 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61eb0fc-1505-43cc-a12b-bf2406586f0a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:59.499788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-a81e2a03-0a08-465a-8051-acb3d8d6ec58 None None] MFA Rules not processed for user `d095ed814b9e47e08d6b5a4fd2901c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:59.501413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a81e2a03-0a08-465a-8051-acb3d8d6ec58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:59.544482 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61eb0fc-1505-43cc-a12b-bf2406586f0a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:31:59.552533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc176b7-8d35-47ee-a3ee-53b21b34b316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:31:59.553443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc176b7-8d35-47ee-a3ee-53b21b34b316 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:59.554420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc176b7-8d35-47ee-a3ee-53b21b34b316 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:59.555040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc176b7-8d35-47ee-a3ee-53b21b34b316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:59.692303 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 882/1745] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:31:59.695540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec78de-a097-4bff-88a3-62375fc0b3b7 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:31:59.697278 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec78de-a097-4bff-88a3-62375fc0b3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:31:59.697720 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec78de-a097-4bff-88a3-62375fc0b3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:31:59.702276 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc176b7-8d35-47ee-a3ee-53b21b34b316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:31:59.705553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc176b7-8d35-47ee-a3ee-53b21b34b316 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:31:59.751931 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 864/1746] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:59 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:31:59.753569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc176b7-8d35-47ee-a3ee-53b21b34b316 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:31:59.763564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cbc176b7-8d35-47ee-a3ee-53b21b34b316 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:59.764210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-89ec78de-a097-4bff-88a3-62375fc0b3b7 None None] MFA Rules not processed for user `36e9c4d95afa44488f82a1110c8571f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:31:59.768033 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-89ec78de-a097-4bff-88a3-62375fc0b3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:31:59.998170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:00.020172 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 883/1747] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:31:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:00.027082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec78de-a097-4bff-88a3-62375fc0b3b7 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:00.143547 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc176b7-8d35-47ee-a3ee-53b21b34b316 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:00.201984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41123271-1f23-4995-9e60-78ccc238fd25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784b6147de91435292e4cf3940bcad9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:00.202826 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41123271-1f23-4995-9e60-78ccc238fd25 service placement] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:00.203137 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41123271-1f23-4995-9e60-78ccc238fd25 service placement] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:00.203296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41123271-1f23-4995-9e60-78ccc238fd25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:00.211671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-41123271-1f23-4995-9e60-78ccc238fd25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:00.275208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41123271-1f23-4995-9e60-78ccc238fd25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:00.292970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41123271-1f23-4995-9e60-78ccc238fd25 service placement] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:00.350750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec78de-a097-4bff-88a3-62375fc0b3b7 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:00.360719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 865/1748] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:31:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:00.406292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b69e1b-8e94-469b-9021-66689dfb0a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:00.415313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b69e1b-8e94-469b-9021-66689dfb0a76 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:00.415613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b69e1b-8e94-469b-9021-66689dfb0a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:00.415911 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b69e1b-8e94-469b-9021-66689dfb0a76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:00.416592 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b69e1b-8e94-469b-9021-66689dfb0a76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:00.434080 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b69e1b-8e94-469b-9021-66689dfb0a76 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:00.562085 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 884/1749] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:00 2022] POST /identity/v3/projects => generated 373 bytes in 541 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:00.562605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41123271-1f23-4995-9e60-78ccc238fd25 service placement] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:00.571337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:00.584509 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41123271-1f23-4995-9e60-78ccc238fd25 service placement] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:00.664058 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b69e1b-8e94-469b-9021-66689dfb0a76 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:00.781260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b4422-f530-4124-9295-840c4a91876d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:00.782243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b4422-f530-4124-9295-840c4a91876d admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:00.782676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b4422-f530-4124-9295-840c4a91876d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:00.782999 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b4422-f530-4124-9295-840c4a91876d admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:00.783638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b4422-f530-4124-9295-840c4a91876d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:00.786229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b4422-f530-4124-9295-840c4a91876d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:00.856288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 866/1750] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:00 2022] POST /identity/v3/users => generated 293 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:32:00.857832 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b4422-f530-4124-9295-840c4a91876d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:00.892364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b4422-f530-4124-9295-840c4a91876d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:00.907595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2abcc-91c4-4ff5-aa3c-c0bb13f75015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:00.908324 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2abcc-91c4-4ff5-aa3c-c0bb13f75015 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:00.908507 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2abcc-91c4-4ff5-aa3c-c0bb13f75015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:00.908778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2abcc-91c4-4ff5-aa3c-c0bb13f75015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:00.911169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2abcc-91c4-4ff5-aa3c-c0bb13f75015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:00.914135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b69e1b-8e94-469b-9021-66689dfb0a76 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:00.914704 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2abcc-91c4-4ff5-aa3c-c0bb13f75015 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:00.951100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abbfd5-3e8a-4fce-adcc-d8d4cf9b3c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:00.951784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abbfd5-3e8a-4fce-adcc-d8d4cf9b3c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:00.952065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abbfd5-3e8a-4fce-adcc-d8d4cf9b3c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:00.952337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abbfd5-3e8a-4fce-adcc-d8d4cf9b3c08 admin admin] PATH_INFO: `/v3/users/28111b0c295541b78cf67ba86035c36c` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:00.954596 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06abbfd5-3e8a-4fce-adcc-d8d4cf9b3c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28111b0c295541b78cf67ba86035c36c)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:00.956766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06abbfd5-3e8a-4fce-adcc-d8d4cf9b3c08 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:00.958368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 867/1751] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:00.968519 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f81cf-ad0e-4a14-a369-3a3b980a19fe None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:00.970406 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f81cf-ad0e-4a14-a369-3a3b980a19fe None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:00.971593 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f81cf-ad0e-4a14-a369-3a3b980a19fe None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:00.977394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 868/1752] 10.222.0.23 () {54 vars in 912 bytes} [Fri Sep 9 16:32:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:32:00.980444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f81cf-ad0e-4a14-a369-3a3b980a19fe None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:01.031132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f81cf-ad0e-4a14-a369-3a3b980a19fe None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:01.066126 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146564f9-90f8-408f-9c58-4a1a3c6ca355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:01.066911 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146564f9-90f8-408f-9c58-4a1a3c6ca355 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:01.067188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146564f9-90f8-408f-9c58-4a1a3c6ca355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:01.067512 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146564f9-90f8-408f-9c58-4a1a3c6ca355 admin admin] PATH_INFO: `/v3/projects/f8b01b9712f34132a05dd7f9eea34814/users/39102350c13d4c66a4f1113fc4093b4a/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:01.089948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-06abbfd5-3e8a-4fce-adcc-d8d4cf9b3c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28111b0c295541b78cf67ba86035c36c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:01.091063 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-06abbfd5-3e8a-4fce-adcc-d8d4cf9b3c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28111b0c295541b78cf67ba86035c36c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:01.092440 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-06abbfd5-3e8a-4fce-adcc-d8d4cf9b3c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28111b0c295541b78cf67ba86035c36c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:01.104745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-06abbfd5-3e8a-4fce-adcc-d8d4cf9b3c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28111b0c295541b78cf67ba86035c36c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:01.132278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146564f9-90f8-408f-9c58-4a1a3c6ca355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b01b9712f34132a05dd7f9eea34814, user_id=39102350c13d4c66a4f1113fc4093b4a, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:01.139691 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146564f9-90f8-408f-9c58-4a1a3c6ca355 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:01.150794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-06abbfd5-3e8a-4fce-adcc-d8d4cf9b3c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28111b0c295541b78cf67ba86035c36c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:01.175463 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 869/1753] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:00 2022] PUT /identity/v3/projects/f8b01b9712f34132a05dd7f9eea34814/users/39102350c13d4c66a4f1113fc4093b4a/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:01.178794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5646ad7-c233-4e14-944a-53a776744c48 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:01.180210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5646ad7-c233-4e14-944a-53a776744c48 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:01.180546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5646ad7-c233-4e14-944a-53a776744c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:01.202863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 885/1754] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:00 2022] DELETE /identity/v3/users/28111b0c295541b78cf67ba86035c36c => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:01.204078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abbfd5-3e8a-4fce-adcc-d8d4cf9b3c08 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:01.206103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-06abbfd5-3e8a-4fce-adcc-d8d4cf9b3c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:01.215827 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:01.263626 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-d5646ad7-c233-4e14-944a-53a776744c48 None None] MFA Rules not processed for user `aa65b6854dc54e289cc3e8e0b3746dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:01.273779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-d5646ad7-c233-4e14-944a-53a776744c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:01.367747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 870/1755] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:32:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:01.369703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3aae8-573d-4eee-a87c-40c0b3367491 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:01.370285 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3aae8-573d-4eee-a87c-40c0b3367491 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:01.371228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3aae8-573d-4eee-a87c-40c0b3367491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:01.372896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abbfd5-3e8a-4fce-adcc-d8d4cf9b3c08 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:01.383394 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92540fa9-eafc-4611-8a88-8b660f4ba0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:01.384138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92540fa9-eafc-4611-8a88-8b660f4ba0b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:01.385092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92540fa9-eafc-4611-8a88-8b660f4ba0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:01.385477 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92540fa9-eafc-4611-8a88-8b660f4ba0b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:01.394623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92540fa9-eafc-4611-8a88-8b660f4ba0b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:01.396408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92540fa9-eafc-4611-8a88-8b660f4ba0b0 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:01.404157 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-cea3aae8-573d-4eee-a87c-40c0b3367491 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:01.407147 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cea3aae8-573d-4eee-a87c-40c0b3367491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:01.412847 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 886/1756] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:01.417139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92540fa9-eafc-4611-8a88-8b660f4ba0b0 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:01.423647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 871/1757] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:01.437728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92540fa9-eafc-4611-8a88-8b660f4ba0b0 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:01.443492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6060c0b-4184-4921-b1e2-7d1f38003432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:01.445317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6060c0b-4184-4921-b1e2-7d1f38003432 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:01.452881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6060c0b-4184-4921-b1e2-7d1f38003432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:01.454784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6060c0b-4184-4921-b1e2-7d1f38003432 admin admin] PATH_INFO: `/v3/projects/f8b01b9712f34132a05dd7f9eea34814/users/39102350c13d4c66a4f1113fc4093b4a/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:01.458729 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6060c0b-4184-4921-b1e2-7d1f38003432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b01b9712f34132a05dd7f9eea34814, user_id=39102350c13d4c66a4f1113fc4093b4a, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:01.465850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6060c0b-4184-4921-b1e2-7d1f38003432 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:01.486526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3aae8-573d-4eee-a87c-40c0b3367491 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:01.489911 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cea3aae8-573d-4eee-a87c-40c0b3367491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:01.518173 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3aae8-573d-4eee-a87c-40c0b3367491 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:01.526527 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 887/1758] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:01 2022] PUT /identity/v3/projects/f8b01b9712f34132a05dd7f9eea34814/users/39102350c13d4c66a4f1113fc4093b4a/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:01.528791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defce388-8567-493f-a811-e9a13377b451 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:01.529083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defce388-8567-493f-a811-e9a13377b451 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:01.533447 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defce388-8567-493f-a811-e9a13377b451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:01.650689 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-defce388-8567-493f-a811-e9a13377b451 None None] MFA Rules not processed for user `39102350c13d4c66a4f1113fc4093b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:01.655378 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-defce388-8567-493f-a811-e9a13377b451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:01.674072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deba506-59c3-4d2d-b75a-3a62e03fe20b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:01.676847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deba506-59c3-4d2d-b75a-3a62e03fe20b service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:01.676847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deba506-59c3-4d2d-b75a-3a62e03fe20b service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:01.676847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deba506-59c3-4d2d-b75a-3a62e03fe20b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:01.738903 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:01.872884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deba506-59c3-4d2d-b75a-3a62e03fe20b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:01.878804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deba506-59c3-4d2d-b75a-3a62e03fe20b service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:01.914086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 872/1759] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:01.916217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deba506-59c3-4d2d-b75a-3a62e03fe20b service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:01.920302 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0deba506-59c3-4d2d-b75a-3a62e03fe20b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:01.932444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 888/1760] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:01.940465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defce388-8567-493f-a811-e9a13377b451 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:01.944516 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-defce388-8567-493f-a811-e9a13377b451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:01.961275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deba506-59c3-4d2d-b75a-3a62e03fe20b service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:01.973454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0f902-5290-44f6-b608-821e03b75c57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:01.974106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0f902-5290-44f6-b608-821e03b75c57 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:01.974385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0f902-5290-44f6-b608-821e03b75c57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:01.974678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0f902-5290-44f6-b608-821e03b75c57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:01.980000 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defce388-8567-493f-a811-e9a13377b451 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:01.993314 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a8d24-ffe6-433d-b3c9-5da8b6f19c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:01.994064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a8d24-ffe6-433d-b3c9-5da8b6f19c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:01.994214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a8d24-ffe6-433d-b3c9-5da8b6f19c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:01.994465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a8d24-ffe6-433d-b3c9-5da8b6f19c52 admin admin] PATH_INFO: `/v3/users/250dcdef3e234a438e0d0aaa6fbdfaab` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:01.997507 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a8d24-ffe6-433d-b3c9-5da8b6f19c52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=250dcdef3e234a438e0d0aaa6fbdfaab)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:01.999205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0f902-5290-44f6-b608-821e03b75c57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:01.999442 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a8d24-ffe6-433d-b3c9-5da8b6f19c52 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:02.000803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0f902-5290-44f6-b608-821e03b75c57 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:02.021840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 873/1761] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:02.023369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621afd53-c596-492c-a57e-7be18a84d212 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:02.024221 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621afd53-c596-492c-a57e-7be18a84d212 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:02.027189 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621afd53-c596-492c-a57e-7be18a84d212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:02.075530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-621afd53-c596-492c-a57e-7be18a84d212 None None] MFA Rules not processed for user `53c0721dc18343b18993ff5af098841d`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:02.076881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-621afd53-c596-492c-a57e-7be18a84d212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:02.133094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-881a8d24-ffe6-433d-b3c9-5da8b6f19c52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '250dcdef3e234a438e0d0aaa6fbdfaab'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:02.136369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-881a8d24-ffe6-433d-b3c9-5da8b6f19c52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '250dcdef3e234a438e0d0aaa6fbdfaab'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:02.138782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-881a8d24-ffe6-433d-b3c9-5da8b6f19c52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '250dcdef3e234a438e0d0aaa6fbdfaab'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:02.161342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-881a8d24-ffe6-433d-b3c9-5da8b6f19c52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '250dcdef3e234a438e0d0aaa6fbdfaab'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:02.185700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:02.194774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-881a8d24-ffe6-433d-b3c9-5da8b6f19c52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '250dcdef3e234a438e0d0aaa6fbdfaab'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:02.201898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 874/1762] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:02.201898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621afd53-c596-492c-a57e-7be18a84d212 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:02.201898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-621afd53-c596-492c-a57e-7be18a84d212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:02.232198 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 889/1763] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:01 2022] DELETE /identity/v3/users/250dcdef3e234a438e0d0aaa6fbdfaab => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:02.233614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c458fb-83eb-43f5-9eeb-84b64e497d35 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:02.234104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c458fb-83eb-43f5-9eeb-84b64e497d35 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:02.237020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c458fb-83eb-43f5-9eeb-84b64e497d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:02.294492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-20c458fb-83eb-43f5-9eeb-84b64e497d35 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:02.297756 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-20c458fb-83eb-43f5-9eeb-84b64e497d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:02.384750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621afd53-c596-492c-a57e-7be18a84d212 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:02.441956 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 890/1764] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:02.461125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c458fb-83eb-43f5-9eeb-84b64e497d35 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:02.462655 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-20c458fb-83eb-43f5-9eeb-84b64e497d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:02.499703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2a0ca-8406-41d1-990e-7009b054c315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:02.500759 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2a0ca-8406-41d1-990e-7009b054c315 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:02.501249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2a0ca-8406-41d1-990e-7009b054c315 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:02.503321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2a0ca-8406-41d1-990e-7009b054c315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:02.506962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8bd2a0ca-8406-41d1-990e-7009b054c315 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:02.552430 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:02.597899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c458fb-83eb-43f5-9eeb-84b64e497d35 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:02.605467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b5177-edd4-4865-a34c-873086be5e7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:02.606155 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b5177-edd4-4865-a34c-873086be5e7a service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:02.606440 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b5177-edd4-4865-a34c-873086be5e7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:02.606789 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b5177-edd4-4865-a34c-873086be5e7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:02.619868 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b5177-edd4-4865-a34c-873086be5e7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:02.621590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b5177-edd4-4865-a34c-873086be5e7a service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:02.635052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 891/1765] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:02.727734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2a0ca-8406-41d1-990e-7009b054c315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:02.729625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2a0ca-8406-41d1-990e-7009b054c315 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:02.759087 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 875/1766] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:02.777282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b5177-edd4-4865-a34c-873086be5e7a service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:02.778917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-763b5177-edd4-4865-a34c-873086be5e7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:02.819368 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b5177-edd4-4865-a34c-873086be5e7a service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:02.837161 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d05eb7-81c6-4812-836e-f65ae57b026c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:02.839273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d05eb7-81c6-4812-836e-f65ae57b026c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:02.840541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d05eb7-81c6-4812-836e-f65ae57b026c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:02.840853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d05eb7-81c6-4812-836e-f65ae57b026c admin admin] PATH_INFO: `/v3/projects/c803181092054dc1ab3a60554871c67a` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:02.851558 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d05eb7-81c6-4812-836e-f65ae57b026c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c803181092054dc1ab3a60554871c67a)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:02.861936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d05eb7-81c6-4812-836e-f65ae57b026c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:03.019311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-a9d05eb7-81c6-4812-836e-f65ae57b026c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c803181092054dc1ab3a60554871c67a', 'request_id': 'req-a9d05eb7-81c6-4812-836e-f65ae57b026c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:03.033134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-a9d05eb7-81c6-4812-836e-f65ae57b026c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c803181092054dc1ab3a60554871c67a', 'request_id': 'req-a9d05eb7-81c6-4812-836e-f65ae57b026c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:03.058641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-a9d05eb7-81c6-4812-836e-f65ae57b026c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c803181092054dc1ab3a60554871c67a', 'request_id': 'req-a9d05eb7-81c6-4812-836e-f65ae57b026c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:03.068346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-a9d05eb7-81c6-4812-836e-f65ae57b026c admin admin] The token cache is being invalidate because project c803181092054dc1ab3a60554871c67a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:03.068644 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-a9d05eb7-81c6-4812-836e-f65ae57b026c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:03.069362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-a9d05eb7-81c6-4812-836e-f65ae57b026c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:03.071441 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 892/1767] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:02 2022] DELETE /identity/v3/projects/c803181092054dc1ab3a60554871c67a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:03.077883 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2a0ca-8406-41d1-990e-7009b054c315 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:03.097899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2a0ca-8406-41d1-990e-7009b054c315 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:03.324865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f8091-e087-43f9-8068-d1ad69878da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:03.325749 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f8091-e087-43f9-8068-d1ad69878da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:03.326225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f8091-e087-43f9-8068-d1ad69878da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:03.326674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f8091-e087-43f9-8068-d1ad69878da5 admin admin] PATH_INFO: `/v3/users/9217b3d8de9449208e1b9298a30ca009` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:03.328034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f8091-e087-43f9-8068-d1ad69878da5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9217b3d8de9449208e1b9298a30ca009)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:03.330739 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f8091-e087-43f9-8068-d1ad69878da5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:03.452917 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a7f8091-e087-43f9-8068-d1ad69878da5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9217b3d8de9449208e1b9298a30ca009'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:03.455741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a7f8091-e087-43f9-8068-d1ad69878da5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9217b3d8de9449208e1b9298a30ca009'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:03.457645 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a7f8091-e087-43f9-8068-d1ad69878da5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9217b3d8de9449208e1b9298a30ca009'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:03.510215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a7f8091-e087-43f9-8068-d1ad69878da5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9217b3d8de9449208e1b9298a30ca009'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:03.576144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a7f8091-e087-43f9-8068-d1ad69878da5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9217b3d8de9449208e1b9298a30ca009'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:03.615912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 876/1768] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:03 2022] DELETE /identity/v3/users/9217b3d8de9449208e1b9298a30ca009 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:03.762329 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d20efd-7a4c-44d4-9e1a-3d67149564c2 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:03.763209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d20efd-7a4c-44d4-9e1a-3d67149564c2 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:03.763485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d20efd-7a4c-44d4-9e1a-3d67149564c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:03.813589 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-c0d20efd-7a4c-44d4-9e1a-3d67149564c2 None None] MFA Rules not processed for user `67d86f09c6304121be2021cb5c15e63e`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:03.819959 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c0d20efd-7a4c-44d4-9e1a-3d67149564c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:03.835212 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:03.883135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f8091-e087-43f9-8068-d1ad69878da5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:03.885204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6a7f8091-e087-43f9-8068-d1ad69878da5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:04.027867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 893/1769] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:04.061308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:04.139124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d20efd-7a4c-44d4-9e1a-3d67149564c2 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:04.141742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c0d20efd-7a4c-44d4-9e1a-3d67149564c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:04.215455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f8091-e087-43f9-8068-d1ad69878da5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:04.230384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed9623-d770-4813-a5f2-5f4c417f87dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:04.231174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed9623-d770-4813-a5f2-5f4c417f87dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:04.231764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed9623-d770-4813-a5f2-5f4c417f87dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:04.235907 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed9623-d770-4813-a5f2-5f4c417f87dc admin admin] PATH_INFO: `/v3/users/36e9c4d95afa44488f82a1110c8571f7` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:04.237018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed9623-d770-4813-a5f2-5f4c417f87dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e9c4d95afa44488f82a1110c8571f7)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:04.243322 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed9623-d770-4813-a5f2-5f4c417f87dc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:04.369577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d20efd-7a4c-44d4-9e1a-3d67149564c2 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:04.383739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:04.386651 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:04.387099 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:04.387902 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] PATH_INFO: `/v3/projects/d4daa77b96154ac480e09962d6035b49` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:04.390438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4daa77b96154ac480e09962d6035b49)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:04.398609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:04.415214 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-06ed9623-d770-4813-a5f2-5f4c417f87dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e9c4d95afa44488f82a1110c8571f7'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:04.416064 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-06ed9623-d770-4813-a5f2-5f4c417f87dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e9c4d95afa44488f82a1110c8571f7'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:04.418671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-06ed9623-d770-4813-a5f2-5f4c417f87dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e9c4d95afa44488f82a1110c8571f7'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:04.442926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-06ed9623-d770-4813-a5f2-5f4c417f87dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e9c4d95afa44488f82a1110c8571f7'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:04.479595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-06ed9623-d770-4813-a5f2-5f4c417f87dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e9c4d95afa44488f82a1110c8571f7'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:04.522437 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4daa77b96154ac480e09962d6035b49', 'request_id': 'req-9a82df61-3c32-4280-b043-1378d27fe6d7'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:04.527241 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 877/1770] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:03 2022] DELETE /identity/v3/users/36e9c4d95afa44488f82a1110c8571f7 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:04.547816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d669e21-1dc8-49e9-a90e-3830382a62f8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:04.547816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d669e21-1dc8-49e9-a90e-3830382a62f8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:04.548148 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d669e21-1dc8-49e9-a90e-3830382a62f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:04.556817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4daa77b96154ac480e09962d6035b49', 'request_id': 'req-9a82df61-3c32-4280-b043-1378d27fe6d7'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:04.593004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4daa77b96154ac480e09962d6035b49', 'request_id': 'req-9a82df61-3c32-4280-b043-1378d27fe6d7'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:04.629197 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] The token cache is being invalidate because project d4daa77b96154ac480e09962d6035b49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:04.629483 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:04.630399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5d669e21-1dc8-49e9-a90e-3830382a62f8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:04.631695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5d669e21-1dc8-49e9-a90e-3830382a62f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:04.634318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:04.647473 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 894/1771] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:04 2022] DELETE /identity/v3/projects/d4daa77b96154ac480e09962d6035b49 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:04.650917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:04.655375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:04.779716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:04.819580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82df61-3c32-4280-b043-1378d27fe6d7 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:04.849271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 878/1772] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:04.856918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deca61-f020-48d1-9f76-2c6ab5477619 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:04.857411 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deca61-f020-48d1-9f76-2c6ab5477619 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:04.857821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deca61-f020-48d1-9f76-2c6ab5477619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:04.947685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:04.948590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:04.948968 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:04.949337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] PATH_INFO: `/v3/projects/8ed6fc8b91ff468895970f2d99e84b70` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:04.950652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ed6fc8b91ff468895970f2d99e84b70)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:04.953231 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:05.002687 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b2deca61-f020-48d1-9f76-2c6ab5477619 None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:05.005014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b2deca61-f020-48d1-9f76-2c6ab5477619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:05.050406 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:05.056132 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ed6fc8b91ff468895970f2d99e84b70', 'request_id': 'req-c616b463-83b6-437e-a56e-fcf7b92e91cf'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:05.108071 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ed6fc8b91ff468895970f2d99e84b70', 'request_id': 'req-c616b463-83b6-437e-a56e-fcf7b92e91cf'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:05.121642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ed6fc8b91ff468895970f2d99e84b70', 'request_id': 'req-c616b463-83b6-437e-a56e-fcf7b92e91cf'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:05.148595 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 879/1773] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:32:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:05.149700 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2deca61-f020-48d1-9f76-2c6ab5477619 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:05.153643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b2deca61-f020-48d1-9f76-2c6ab5477619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:05.153880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] The token cache is being invalidate because project 8ed6fc8b91ff468895970f2d99e84b70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:05.154282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:05.155120 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:05.162846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 895/1774] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:04 2022] DELETE /identity/v3/projects/8ed6fc8b91ff468895970f2d99e84b70 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:05.164907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:05.166686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:05.180178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2deca61-f020-48d1-9f76-2c6ab5477619 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:05.291577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616b463-83b6-437e-a56e-fcf7b92e91cf admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:05.296501 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c77a9-a82d-45fa-a30b-6c8bd3f1b468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:05.298898 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c77a9-a82d-45fa-a30b-6c8bd3f1b468 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:05.299664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c77a9-a82d-45fa-a30b-6c8bd3f1b468 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:05.300147 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c77a9-a82d-45fa-a30b-6c8bd3f1b468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:05.302029 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048cc18-0262-408b-a2ce-e3f1f5a309aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:05.302729 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048cc18-0262-408b-a2ce-e3f1f5a309aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:05.303030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048cc18-0262-408b-a2ce-e3f1f5a309aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:05.303298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048cc18-0262-408b-a2ce-e3f1f5a309aa admin admin] PATH_INFO: `/v3/users/14362fb1f81f4cf29256e35caec11afa` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:05.311118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048cc18-0262-408b-a2ce-e3f1f5a309aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=14362fb1f81f4cf29256e35caec11afa)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:05.313380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048cc18-0262-408b-a2ce-e3f1f5a309aa admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:05.316675 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-724c77a9-a82d-45fa-a30b-6c8bd3f1b468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:05.351493 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c77a9-a82d-45fa-a30b-6c8bd3f1b468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:05.353188 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c77a9-a82d-45fa-a30b-6c8bd3f1b468 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:05.386473 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 896/1775] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:05.407262 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c734bc2-74fa-4d20-9d13-c8bc426a9df8 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:05.412855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c734bc2-74fa-4d20-9d13-c8bc426a9df8 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:05.416853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c734bc2-74fa-4d20-9d13-c8bc426a9df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:05.432265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d048cc18-0262-408b-a2ce-e3f1f5a309aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14362fb1f81f4cf29256e35caec11afa'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:05.434036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d048cc18-0262-408b-a2ce-e3f1f5a309aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14362fb1f81f4cf29256e35caec11afa'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:05.436303 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d048cc18-0262-408b-a2ce-e3f1f5a309aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14362fb1f81f4cf29256e35caec11afa'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:05.469214 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d048cc18-0262-408b-a2ce-e3f1f5a309aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14362fb1f81f4cf29256e35caec11afa'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:05.496434 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-9c734bc2-74fa-4d20-9d13-c8bc426a9df8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:05.496749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-9c734bc2-74fa-4d20-9d13-c8bc426a9df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:05.515523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d048cc18-0262-408b-a2ce-e3f1f5a309aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14362fb1f81f4cf29256e35caec11afa'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:05.541547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 897/1776] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:05.555504 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c734bc2-74fa-4d20-9d13-c8bc426a9df8 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:05.556952 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 880/1777] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:05 2022] DELETE /identity/v3/users/14362fb1f81f4cf29256e35caec11afa => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:05.558893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fe8e7-a24b-4dd6-8e4d-5229d8c9c124 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:05.559261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fe8e7-a24b-4dd6-8e4d-5229d8c9c124 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:05.559557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fe8e7-a24b-4dd6-8e4d-5229d8c9c124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:05.560281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-9c734bc2-74fa-4d20-9d13-c8bc426a9df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:05.618629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1f7fe8e7-a24b-4dd6-8e4d-5229d8c9c124 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:05.619811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1f7fe8e7-a24b-4dd6-8e4d-5229d8c9c124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:05.728058 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:05.779747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 881/1778] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:05.780481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:05.784293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fe8e7-a24b-4dd6-8e4d-5229d8c9c124 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:05.800125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fe8e7-a24b-4dd6-8e4d-5229d8c9c124 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:05.814307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16745ac9-cfb0-4ef9-9ed4-b794b65c0d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:05.816662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16745ac9-cfb0-4ef9-9ed4-b794b65c0d88 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:05.816948 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16745ac9-cfb0-4ef9-9ed4-b794b65c0d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:05.817246 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16745ac9-cfb0-4ef9-9ed4-b794b65c0d88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:05.817921 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16745ac9-cfb0-4ef9-9ed4-b794b65c0d88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:05.818630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16745ac9-cfb0-4ef9-9ed4-b794b65c0d88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:05.819937 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16745ac9-cfb0-4ef9-9ed4-b794b65c0d88 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:05.847758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 882/1779] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:32:05.852281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16745ac9-cfb0-4ef9-9ed4-b794b65c0d88 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:05.870539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16745ac9-cfb0-4ef9-9ed4-b794b65c0d88 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:05.896045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba0a17-5dd9-4671-85c7-7df892aa1628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:05.896671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba0a17-5dd9-4671-85c7-7df892aa1628 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:05.896952 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba0a17-5dd9-4671-85c7-7df892aa1628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:05.897223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba0a17-5dd9-4671-85c7-7df892aa1628 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:05.897999 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba0a17-5dd9-4671-85c7-7df892aa1628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:05.900507 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba0a17-5dd9-4671-85c7-7df892aa1628 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:05.991034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 883/1780] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:05 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:32:05.992634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba0a17-5dd9-4671-85c7-7df892aa1628 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:06.107615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c734bc2-74fa-4d20-9d13-c8bc426a9df8 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:06.230946 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba0a17-5dd9-4671-85c7-7df892aa1628 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:06.248798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee437d5-f9ff-45f0-bec4-c04be7cde712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:06.251778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee437d5-f9ff-45f0-bec4-c04be7cde712 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:06.253704 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee437d5-f9ff-45f0-bec4-c04be7cde712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:06.254034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee437d5-f9ff-45f0-bec4-c04be7cde712 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:06.255075 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee437d5-f9ff-45f0-bec4-c04be7cde712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:06.259353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee437d5-f9ff-45f0-bec4-c04be7cde712 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:06.337541 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 884/1781] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:05 2022] POST /identity/v3/users => generated 288 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:06.340107 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee437d5-f9ff-45f0-bec4-c04be7cde712 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:06.340952 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa041d-44d3-4673-af3c-623095103bba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:06.341912 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa041d-44d3-4673-af3c-623095103bba service swift] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:06.345401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa041d-44d3-4673-af3c-623095103bba service swift] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:06.346737 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa041d-44d3-4673-af3c-623095103bba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:06.352996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cdfa041d-44d3-4673-af3c-623095103bba service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:06.374493 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee437d5-f9ff-45f0-bec4-c04be7cde712 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:06.423541 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bed8a8-a635-457b-a90b-b120e3848d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:06.424177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bed8a8-a635-457b-a90b-b120e3848d10 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:06.424826 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bed8a8-a635-457b-a90b-b120e3848d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:06.425098 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bed8a8-a635-457b-a90b-b120e3848d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:06.425847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bed8a8-a635-457b-a90b-b120e3848d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:06.427545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bed8a8-a635-457b-a90b-b120e3848d10 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:06.495985 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 885/1782] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:06.497057 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bed8a8-a635-457b-a90b-b120e3848d10 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:06.514352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bed8a8-a635-457b-a90b-b120e3848d10 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:06.535212 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e686675-d74a-4b65-aca8-e91b9668af1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:06.537027 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e686675-d74a-4b65-aca8-e91b9668af1a admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:06.537309 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e686675-d74a-4b65-aca8-e91b9668af1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:06.537589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e686675-d74a-4b65-aca8-e91b9668af1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:06.541152 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e686675-d74a-4b65-aca8-e91b9668af1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:06.547861 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e686675-d74a-4b65-aca8-e91b9668af1a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:06.583494 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 886/1783] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:06.587324 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e686675-d74a-4b65-aca8-e91b9668af1a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:06.608761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e686675-d74a-4b65-aca8-e91b9668af1a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:06.628345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa041d-44d3-4673-af3c-623095103bba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:06.628979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11752381-3462-4475-a1c3-1d4dc5959ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:06.629830 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11752381-3462-4475-a1c3-1d4dc5959ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:06.630271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11752381-3462-4475-a1c3-1d4dc5959ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:06.630381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa041d-44d3-4673-af3c-623095103bba service swift] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:06.630795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11752381-3462-4475-a1c3-1d4dc5959ad3 admin admin] PATH_INFO: `/v3/projects/ea8147c9ea604f3783001b054aee6786/users/67dc89a481ff44ea86f4719de8186b72/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:06.670480 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11752381-3462-4475-a1c3-1d4dc5959ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea8147c9ea604f3783001b054aee6786, user_id=67dc89a481ff44ea86f4719de8186b72, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:06.674143 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11752381-3462-4475-a1c3-1d4dc5959ad3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:06.680494 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 898/1784] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:32:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:32:06.711310 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 887/1785] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:06 2022] PUT /identity/v3/projects/ea8147c9ea604f3783001b054aee6786/users/67dc89a481ff44ea86f4719de8186b72/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:06.722976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3002f4c-8dc9-481e-9c0b-1ecfb70809b9 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:06.723659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3002f4c-8dc9-481e-9c0b-1ecfb70809b9 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:06.724143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3002f4c-8dc9-481e-9c0b-1ecfb70809b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:06.787006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-e3002f4c-8dc9-481e-9c0b-1ecfb70809b9 None None] MFA Rules not processed for user `67dc89a481ff44ea86f4719de8186b72`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:06.790734 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e3002f4c-8dc9-481e-9c0b-1ecfb70809b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:06.915491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:07.007484 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 899/1786] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:07.011772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11752381-3462-4475-a1c3-1d4dc5959ad3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:07.062484 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3002f4c-8dc9-481e-9c0b-1ecfb70809b9 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:07.064578 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e3002f4c-8dc9-481e-9c0b-1ecfb70809b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:07.163181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:07.194825 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11752381-3462-4475-a1c3-1d4dc5959ad3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:07.211308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfb294-5761-4cbb-a61e-a7f9abdf6a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:07.211958 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfb294-5761-4cbb-a61e-a7f9abdf6a63 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:07.212235 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfb294-5761-4cbb-a61e-a7f9abdf6a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:07.212501 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfb294-5761-4cbb-a61e-a7f9abdf6a63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:07.213149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfb294-5761-4cbb-a61e-a7f9abdf6a63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:07.236647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfb294-5761-4cbb-a61e-a7f9abdf6a63 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:07.257703 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3002f4c-8dc9-481e-9c0b-1ecfb70809b9 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:07.279576 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:07.280744 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:07.281277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:07.282311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] PATH_INFO: `/v3/projects/940ba6cb0f2f47b98a21a9816e16d2d9` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:07.286091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=940ba6cb0f2f47b98a21a9816e16d2d9)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:07.291672 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:07.372798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 888/1787] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:07 2022] POST /identity/v3/projects => generated 375 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:07.372798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfb294-5761-4cbb-a61e-a7f9abdf6a63 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:07.425946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '940ba6cb0f2f47b98a21a9816e16d2d9', 'request_id': 'req-e462c388-94b1-4c20-8b70-5fb8c405c1f4'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:07.437759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '940ba6cb0f2f47b98a21a9816e16d2d9', 'request_id': 'req-e462c388-94b1-4c20-8b70-5fb8c405c1f4'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:07.452975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '940ba6cb0f2f47b98a21a9816e16d2d9', 'request_id': 'req-e462c388-94b1-4c20-8b70-5fb8c405c1f4'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:07.502645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] The token cache is being invalidate because project 940ba6cb0f2f47b98a21a9816e16d2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:07.503086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:07.504326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:07.507944 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 900/1788] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:07 2022] DELETE /identity/v3/projects/940ba6cb0f2f47b98a21a9816e16d2d9 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:07.512547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:07.514596 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:07.585383 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfb294-5761-4cbb-a61e-a7f9abdf6a63 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:07.692798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462c388-94b1-4c20-8b70-5fb8c405c1f4 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:07.709268 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23272970-f704-453c-ac6c-2cdf0a662d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:07.712759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23272970-f704-453c-ac6c-2cdf0a662d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:07.714118 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23272970-f704-453c-ac6c-2cdf0a662d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:07.714918 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23272970-f704-453c-ac6c-2cdf0a662d09 admin admin] PATH_INFO: `/v3/users/39102350c13d4c66a4f1113fc4093b4a` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:07.718442 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23272970-f704-453c-ac6c-2cdf0a662d09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39102350c13d4c66a4f1113fc4093b4a)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:07.725666 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23272970-f704-453c-ac6c-2cdf0a662d09 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:07.737316 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:07.756347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f7f88-0582-4d81-8769-027738c5690f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:07.756987 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f7f88-0582-4d81-8769-027738c5690f admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:07.757266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f7f88-0582-4d81-8769-027738c5690f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:07.757537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f7f88-0582-4d81-8769-027738c5690f admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:07.761506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f7f88-0582-4d81-8769-027738c5690f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:07.767651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f7f88-0582-4d81-8769-027738c5690f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:07.829352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 889/1789] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:07 2022] POST /identity/v3/users => generated 294 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:07.837076 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f7f88-0582-4d81-8769-027738c5690f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:07.840837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b91f7f88-0582-4d81-8769-027738c5690f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:07.853886 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-23272970-f704-453c-ac6c-2cdf0a662d09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39102350c13d4c66a4f1113fc4093b4a'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:07.854810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-23272970-f704-453c-ac6c-2cdf0a662d09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39102350c13d4c66a4f1113fc4093b4a'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:07.856589 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-23272970-f704-453c-ac6c-2cdf0a662d09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39102350c13d4c66a4f1113fc4093b4a'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:07.865601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f7f88-0582-4d81-8769-027738c5690f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:07.872011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-23272970-f704-453c-ac6c-2cdf0a662d09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39102350c13d4c66a4f1113fc4093b4a'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:07.882537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c806d1-1695-4324-bc95-c0e32d195f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:07.883330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c806d1-1695-4324-bc95-c0e32d195f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:07.883608 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c806d1-1695-4324-bc95-c0e32d195f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:07.885487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c806d1-1695-4324-bc95-c0e32d195f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:07.892995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c806d1-1695-4324-bc95-c0e32d195f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:07.892995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c806d1-1695-4324-bc95-c0e32d195f2a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:07.906303 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-23272970-f704-453c-ac6c-2cdf0a662d09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39102350c13d4c66a4f1113fc4093b4a'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:07.914623 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 890/1790] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:07.918945 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c806d1-1695-4324-bc95-c0e32d195f2a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:07.921778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f2c806d1-1695-4324-bc95-c0e32d195f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:07.939548 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 901/1791] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:07 2022] DELETE /identity/v3/users/39102350c13d4c66a4f1113fc4093b4a => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:07.951193 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c806d1-1695-4324-bc95-c0e32d195f2a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:08.240927 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d807e01-e435-471b-9490-f109d4caabe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:08.241785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d807e01-e435-471b-9490-f109d4caabe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:08.242223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d807e01-e435-471b-9490-f109d4caabe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:08.242589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d807e01-e435-471b-9490-f109d4caabe1 admin admin] PATH_INFO: `/v3/projects/d77a194dc2f54e8d8843f361f2088266/users/b83038a71c4542138a72a608507cb51d/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:08.247540 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:08.308818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d807e01-e435-471b-9490-f109d4caabe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d77a194dc2f54e8d8843f361f2088266, user_id=b83038a71c4542138a72a608507cb51d, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:08.318764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d807e01-e435-471b-9490-f109d4caabe1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:08.364729 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 891/1792] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:07 2022] PUT /identity/v3/projects/d77a194dc2f54e8d8843f361f2088266/users/b83038a71c4542138a72a608507cb51d/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:08.370675 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23272970-f704-453c-ac6c-2cdf0a662d09 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:08.405322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23272970-f704-453c-ac6c-2cdf0a662d09 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:08.651132 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96f475-470f-4155-be31-bc7455c80243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:08.651778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96f475-470f-4155-be31-bc7455c80243 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:08.652063 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96f475-470f-4155-be31-bc7455c80243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:08.668420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96f475-470f-4155-be31-bc7455c80243 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:08.669434 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96f475-470f-4155-be31-bc7455c80243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:08.673873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96f475-470f-4155-be31-bc7455c80243 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:08.792045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 902/1793] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:08.796294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d807e01-e435-471b-9490-f109d4caabe1 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:08.816501 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d807e01-e435-471b-9490-f109d4caabe1 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:08.849880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0bfce-5b1f-422f-9f6f-3b1a8a59c492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:08.854601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0bfce-5b1f-422f-9f6f-3b1a8a59c492 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:08.855036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0bfce-5b1f-422f-9f6f-3b1a8a59c492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:08.855320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0bfce-5b1f-422f-9f6f-3b1a8a59c492 admin admin] PATH_INFO: `/v3/projects/d77a194dc2f54e8d8843f361f2088266/users/b83038a71c4542138a72a608507cb51d/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:08.865951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0bfce-5b1f-422f-9f6f-3b1a8a59c492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d77a194dc2f54e8d8843f361f2088266, user_id=b83038a71c4542138a72a608507cb51d, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:08.876361 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0bfce-5b1f-422f-9f6f-3b1a8a59c492 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:08.919905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 892/1794] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:08 2022] PUT /identity/v3/projects/d77a194dc2f54e8d8843f361f2088266/users/b83038a71c4542138a72a608507cb51d/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:08.931955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3c756-a806-43f2-acf0-5bddadbfa251 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:08.931955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3c756-a806-43f2-acf0-5bddadbfa251 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:08.932162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3c756-a806-43f2-acf0-5bddadbfa251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:09.113029 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d7c3c756-a806-43f2-acf0-5bddadbfa251 None None] MFA Rules not processed for user `b83038a71c4542138a72a608507cb51d`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:09.122669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d7c3c756-a806-43f2-acf0-5bddadbfa251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:09.218609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0bfce-5b1f-422f-9f6f-3b1a8a59c492 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:09.221083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b2f0bfce-5b1f-422f-9f6f-3b1a8a59c492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:09.312403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 903/1795] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:09.325255 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3c756-a806-43f2-acf0-5bddadbfa251 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:09.326942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d7c3c756-a806-43f2-acf0-5bddadbfa251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:09.335228 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:09.448255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0bfce-5b1f-422f-9f6f-3b1a8a59c492 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:09.455055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:09.455626 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3c756-a806-43f2-acf0-5bddadbfa251 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:09.456119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:09.456119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:09.456281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57 admin admin] PATH_INFO: `/v3/projects/563ee41fae234711992a8d9289a9984d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:09.458352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=563ee41fae234711992a8d9289a9984d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:09.460655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:09.461304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbe70a-a1bc-407f-99e5-89bf966bd27d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:09.462717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbe70a-a1bc-407f-99e5-89bf966bd27d service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:09.463025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbe70a-a1bc-407f-99e5-89bf966bd27d service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:09.463400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbe70a-a1bc-407f-99e5-89bf966bd27d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:09.466100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-16dbe70a-a1bc-407f-99e5-89bf966bd27d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:09.555221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbe70a-a1bc-407f-99e5-89bf966bd27d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:09.557529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbe70a-a1bc-407f-99e5-89bf966bd27d service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:09.569171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 904/1796] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:09.665619 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '563ee41fae234711992a8d9289a9984d', 'request_id': 'req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:09.680606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '563ee41fae234711992a8d9289a9984d', 'request_id': 'req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:09.693610 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '563ee41fae234711992a8d9289a9984d', 'request_id': 'req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:09.720871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57 admin admin] The token cache is being invalidate because project 563ee41fae234711992a8d9289a9984d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:09.721256 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:09.722206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c7f590b9-0d9f-4fa4-850d-9c86edc61f57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:09.729269 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 893/1797] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:09 2022] DELETE /identity/v3/projects/563ee41fae234711992a8d9289a9984d => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:09.737715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efbe2b-f775-4fb6-b404-60382b321e57 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:09.738528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efbe2b-f775-4fb6-b404-60382b321e57 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:09.740632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efbe2b-f775-4fb6-b404-60382b321e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:09.781480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-41efbe2b-f775-4fb6-b404-60382b321e57 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:09.783229 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-41efbe2b-f775-4fb6-b404-60382b321e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:09.947809 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 905/1798] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:09.958698 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008f1b6-15e4-435e-957b-8fddb302fc6e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:09.958985 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008f1b6-15e4-435e-957b-8fddb302fc6e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:09.959399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008f1b6-15e4-435e-957b-8fddb302fc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:10.064138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1008f1b6-15e4-435e-957b-8fddb302fc6e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:10.067086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1008f1b6-15e4-435e-957b-8fddb302fc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:10.116291 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 894/1799] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:10.119970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efbe2b-f775-4fb6-b404-60382b321e57 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:10.137136 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efbe2b-f775-4fb6-b404-60382b321e57 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:10.157700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf4f61-b48f-47e7-8441-c05b995e0cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:10.158661 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf4f61-b48f-47e7-8441-c05b995e0cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:10.159465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf4f61-b48f-47e7-8441-c05b995e0cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:10.159803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf4f61-b48f-47e7-8441-c05b995e0cf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:10.161755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf4f61-b48f-47e7-8441-c05b995e0cf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:10.161755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf4f61-b48f-47e7-8441-c05b995e0cf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:10.194538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf4f61-b48f-47e7-8441-c05b995e0cf2 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:10.279963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 906/1800] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:32:10.286225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008f1b6-15e4-435e-957b-8fddb302fc6e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:10.301460 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008f1b6-15e4-435e-957b-8fddb302fc6e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:10.318989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f02efd-96f7-4a07-858f-682d20c92e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:10.320055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f02efd-96f7-4a07-858f-682d20c92e39 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:10.320496 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f02efd-96f7-4a07-858f-682d20c92e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:10.320918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f02efd-96f7-4a07-858f-682d20c92e39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:10.321701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f02efd-96f7-4a07-858f-682d20c92e39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:10.323808 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f02efd-96f7-4a07-858f-682d20c92e39 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:10.383692 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 895/1801] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:10 2022] POST /identity/v3/projects => generated 359 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:10.385244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf4f61-b48f-47e7-8441-c05b995e0cf2 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:10.400095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf4f61-b48f-47e7-8441-c05b995e0cf2 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:10.468115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f02efd-96f7-4a07-858f-682d20c92e39 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:10.470380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-04f02efd-96f7-4a07-858f-682d20c92e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:10.548513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:10.637588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f02efd-96f7-4a07-858f-682d20c92e39 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:10.669236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2983a-7626-4660-8912-e61083f0faa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:10.669236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2983a-7626-4660-8912-e61083f0faa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:10.669236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2983a-7626-4660-8912-e61083f0faa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:10.669236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2983a-7626-4660-8912-e61083f0faa9 admin admin] PATH_INFO: `/v3/projects/f8b01b9712f34132a05dd7f9eea34814` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:10.670583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2983a-7626-4660-8912-e61083f0faa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8b01b9712f34132a05dd7f9eea34814)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:10.672755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2983a-7626-4660-8912-e61083f0faa9 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:10.680081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:10.706615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea0cd6-b7d8-4d7e-8699-0bf59f63c1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:10.708624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea0cd6-b7d8-4d7e-8699-0bf59f63c1bc admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:10.709210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea0cd6-b7d8-4d7e-8699-0bf59f63c1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:10.710798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea0cd6-b7d8-4d7e-8699-0bf59f63c1bc admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:10.712072 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea0cd6-b7d8-4d7e-8699-0bf59f63c1bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:10.717314 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea0cd6-b7d8-4d7e-8699-0bf59f63c1bc admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:10.827420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 907/1802] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:10 2022] POST /identity/v3/users => generated 280 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:10.828985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea0cd6-b7d8-4d7e-8699-0bf59f63c1bc admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:10.844004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea0cd6-b7d8-4d7e-8699-0bf59f63c1bc admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:10.914060 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-faa2983a-7626-4660-8912-e61083f0faa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b01b9712f34132a05dd7f9eea34814', 'request_id': 'req-faa2983a-7626-4660-8912-e61083f0faa9'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:10.942610 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-faa2983a-7626-4660-8912-e61083f0faa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b01b9712f34132a05dd7f9eea34814', 'request_id': 'req-faa2983a-7626-4660-8912-e61083f0faa9'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:10.987173 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-faa2983a-7626-4660-8912-e61083f0faa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b01b9712f34132a05dd7f9eea34814', 'request_id': 'req-faa2983a-7626-4660-8912-e61083f0faa9'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:11.049896 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-faa2983a-7626-4660-8912-e61083f0faa9 admin admin] The token cache is being invalidate because project f8b01b9712f34132a05dd7f9eea34814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:11.057022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-faa2983a-7626-4660-8912-e61083f0faa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:11.067674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-faa2983a-7626-4660-8912-e61083f0faa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:11.070923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf2b2f-09e4-44ba-bb7b-7689cccf4dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:11.072898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 896/1803] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:10 2022] DELETE /identity/v3/projects/f8b01b9712f34132a05dd7f9eea34814 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:11.074277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf2b2f-09e4-44ba-bb7b-7689cccf4dde admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:11.077731 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fad7ed-bce9-4494-96aa-d7d37c61f66c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:11.078247 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf2b2f-09e4-44ba-bb7b-7689cccf4dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:11.078642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf2b2f-09e4-44ba-bb7b-7689cccf4dde admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:11.079448 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf2b2f-09e4-44ba-bb7b-7689cccf4dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:11.079815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fad7ed-bce9-4494-96aa-d7d37c61f66c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:11.080797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fad7ed-bce9-4494-96aa-d7d37c61f66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:11.083401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf2b2f-09e4-44ba-bb7b-7689cccf4dde admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:11.140554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 908/1804] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:32:11.144305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf2b2f-09e4-44ba-bb7b-7689cccf4dde admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:11.154852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-bbbf2b2f-09e4-44ba-bb7b-7689cccf4dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:11.176202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-79fad7ed-bce9-4494-96aa-d7d37c61f66c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:11.181993 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-79fad7ed-bce9-4494-96aa-d7d37c61f66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:11.196149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf2b2f-09e4-44ba-bb7b-7689cccf4dde admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:11.208161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 897/1805] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:11.212547 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcb9c7-da68-43c4-b020-a74369703c09 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:11.213516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcb9c7-da68-43c4-b020-a74369703c09 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:11.214253 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcb9c7-da68-43c4-b020-a74369703c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:11.221662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a0cfe-bc82-4f72-8109-3a93bf0a483c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:11.222587 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a0cfe-bc82-4f72-8109-3a93bf0a483c admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:11.223464 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a0cfe-bc82-4f72-8109-3a93bf0a483c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:11.223464 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a0cfe-bc82-4f72-8109-3a93bf0a483c admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:11.224803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a0cfe-bc82-4f72-8109-3a93bf0a483c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:11.230261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a0cfe-bc82-4f72-8109-3a93bf0a483c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:11.282646 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 909/1806] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:11.283869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a0cfe-bc82-4f72-8109-3a93bf0a483c admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:11.305362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a0cfe-bc82-4f72-8109-3a93bf0a483c admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:11.310445 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dc14e-6c70-4e7b-944c-315a2757a906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:11.312762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-47dcb9c7-da68-43c4-b020-a74369703c09 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:11.315310 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-47dcb9c7-da68-43c4-b020-a74369703c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:11.315724 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dc14e-6c70-4e7b-944c-315a2757a906 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:11.316329 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dc14e-6c70-4e7b-944c-315a2757a906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:11.316742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dc14e-6c70-4e7b-944c-315a2757a906 admin admin] PATH_INFO: `/v3/projects/6dff9723a7e144268b84c20a815183ef/users/a6a9b3f7ef504dca828555d0e52ed324/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:11.360688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51dc14e-6c70-4e7b-944c-315a2757a906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dff9723a7e144268b84c20a815183ef, user_id=a6a9b3f7ef504dca828555d0e52ed324, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:11.368814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51dc14e-6c70-4e7b-944c-315a2757a906 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:11.388882 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 898/1807] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:11.389981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcb9c7-da68-43c4-b020-a74369703c09 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:11.391640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 910/1808] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:11 2022] PUT /identity/v3/projects/6dff9723a7e144268b84c20a815183ef/users/a6a9b3f7ef504dca828555d0e52ed324/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:11.409221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95685ab-4d9f-4db9-bbff-434c9b3e74c8 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:11.409542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95685ab-4d9f-4db9-bbff-434c9b3e74c8 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:11.409976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95685ab-4d9f-4db9-bbff-434c9b3e74c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:11.428448 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcb9c7-da68-43c4-b020-a74369703c09 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:11.460847 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d95685ab-4d9f-4db9-bbff-434c9b3e74c8 None None] MFA Rules not processed for user `a6a9b3f7ef504dca828555d0e52ed324`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:11.462314 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d95685ab-4d9f-4db9-bbff-434c9b3e74c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:11.611119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e12f2-e7ab-461e-b244-20001e855049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:11.611817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e12f2-e7ab-461e-b244-20001e855049 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:11.612078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e12f2-e7ab-461e-b244-20001e855049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:11.612351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e12f2-e7ab-461e-b244-20001e855049 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:11.613106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e12f2-e7ab-461e-b244-20001e855049 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:11.613413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e12f2-e7ab-461e-b244-20001e855049 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:11.615137 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e12f2-e7ab-461e-b244-20001e855049 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:11.630300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 899/1809] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:32:11.634776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e12f2-e7ab-461e-b244-20001e855049 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:11.652832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 911/1810] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:11.661229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e12f2-e7ab-461e-b244-20001e855049 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:11.666485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d49e5-3e32-453f-850f-9cebcfdef9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:11.667180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d49e5-3e32-453f-850f-9cebcfdef9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:11.667461 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d49e5-3e32-453f-850f-9cebcfdef9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:11.667731 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d49e5-3e32-453f-850f-9cebcfdef9b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:11.668417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d49e5-3e32-453f-850f-9cebcfdef9b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:11.672524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d49e5-3e32-453f-850f-9cebcfdef9b3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:11.698535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95685ab-4d9f-4db9-bbff-434c9b3e74c8 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:11.702680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d95685ab-4d9f-4db9-bbff-434c9b3e74c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:11.743662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 900/1811] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:11 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:11.747697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d49e5-3e32-453f-850f-9cebcfdef9b3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:11.793185 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:11.793279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:11.854418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95685ab-4d9f-4db9-bbff-434c9b3e74c8 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:11.913463 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e4f90-7038-4b5b-bd8a-669107524a31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:11.914087 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e4f90-7038-4b5b-bd8a-669107524a31 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:11.914354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e4f90-7038-4b5b-bd8a-669107524a31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:11.914657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e4f90-7038-4b5b-bd8a-669107524a31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:11.945436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d49e5-3e32-453f-850f-9cebcfdef9b3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:11.959899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748622b-b12e-44e9-947a-aaed27a74990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:11.960903 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748622b-b12e-44e9-947a-aaed27a74990 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:11.961389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748622b-b12e-44e9-947a-aaed27a74990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:11.961863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748622b-b12e-44e9-947a-aaed27a74990 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:11.964365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748622b-b12e-44e9-947a-aaed27a74990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:11.970581 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748622b-b12e-44e9-947a-aaed27a74990 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:12.002911 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e4f90-7038-4b5b-bd8a-669107524a31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:12.004497 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e4f90-7038-4b5b-bd8a-669107524a31 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:12.023874 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 912/1812] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:11 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:12.043019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e4f90-7038-4b5b-bd8a-669107524a31 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:12.054189 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 901/1813] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:11 2022] POST /identity/v3/users => generated 283 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:12.058887 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748622b-b12e-44e9-947a-aaed27a74990 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:12.073547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e4f90-7038-4b5b-bd8a-669107524a31 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:12.079599 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3b43c-665e-46d9-b796-08505e0a50f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:12.080271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3b43c-665e-46d9-b796-08505e0a50f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:12.080548 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3b43c-665e-46d9-b796-08505e0a50f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:12.080816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3b43c-665e-46d9-b796-08505e0a50f5 admin admin] PATH_INFO: `/v3/users/a6a9b3f7ef504dca828555d0e52ed324` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:12.082052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3b43c-665e-46d9-b796-08505e0a50f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6a9b3f7ef504dca828555d0e52ed324)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:12.084052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3b43c-665e-46d9-b796-08505e0a50f5 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:12.100223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748622b-b12e-44e9-947a-aaed27a74990 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:12.110942 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ae573-08b6-4d59-b5fc-482863ec3589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:12.112494 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ae573-08b6-4d59-b5fc-482863ec3589 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:12.113864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ae573-08b6-4d59-b5fc-482863ec3589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:12.114917 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ae573-08b6-4d59-b5fc-482863ec3589 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:12.116403 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ae573-08b6-4d59-b5fc-482863ec3589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:12.120788 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ae573-08b6-4d59-b5fc-482863ec3589 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:12.140372 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 902/1814] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:12.147701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ae573-08b6-4d59-b5fc-482863ec3589 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:12.159031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ae573-08b6-4d59-b5fc-482863ec3589 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:12.165811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ac9bf-37b5-4fb0-a882-6f6e38ad1824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:12.167157 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ac9bf-37b5-4fb0-a882-6f6e38ad1824 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:12.167361 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ac9bf-37b5-4fb0-a882-6f6e38ad1824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:12.167771 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ac9bf-37b5-4fb0-a882-6f6e38ad1824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:12.169246 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ac9bf-37b5-4fb0-a882-6f6e38ad1824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:12.171215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ac9bf-37b5-4fb0-a882-6f6e38ad1824 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:12.188769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 903/1815] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:12.189779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ac9bf-37b5-4fb0-a882-6f6e38ad1824 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:12.191282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-23a3b43c-665e-46d9-b796-08505e0a50f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6a9b3f7ef504dca828555d0e52ed324'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:12.192006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-23a3b43c-665e-46d9-b796-08505e0a50f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6a9b3f7ef504dca828555d0e52ed324'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:12.192631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-23a3b43c-665e-46d9-b796-08505e0a50f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6a9b3f7ef504dca828555d0e52ed324'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:12.203139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-23a3b43c-665e-46d9-b796-08505e0a50f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6a9b3f7ef504dca828555d0e52ed324'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:12.220035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-23a3b43c-665e-46d9-b796-08505e0a50f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6a9b3f7ef504dca828555d0e52ed324'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:12.246015 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ac9bf-37b5-4fb0-a882-6f6e38ad1824 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:12.271184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0b0c1-ad28-4855-9cb5-865c1a10d655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:12.275545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0b0c1-ad28-4855-9cb5-865c1a10d655 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:12.275900 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0b0c1-ad28-4855-9cb5-865c1a10d655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:12.276175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0b0c1-ad28-4855-9cb5-865c1a10d655 admin admin] PATH_INFO: `/v3/projects/670e32b4b2a44d0c83d60258d73bdc9e/users/39e0161d99f04f3697e6f3925310ea26/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:12.279196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 913/1816] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:12 2022] DELETE /identity/v3/users/a6a9b3f7ef504dca828555d0e52ed324 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:12.281164 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c0d89-f3ec-4382-8f50-505e18affa9a None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:12.284610 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c0d89-f3ec-4382-8f50-505e18affa9a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:12.284958 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c0d89-f3ec-4382-8f50-505e18affa9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:12.337850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0b0c1-ad28-4855-9cb5-865c1a10d655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=670e32b4b2a44d0c83d60258d73bdc9e, user_id=39e0161d99f04f3697e6f3925310ea26, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:12.342231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0b0c1-ad28-4855-9cb5-865c1a10d655 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:12.361369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-125c0d89-f3ec-4382-8f50-505e18affa9a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:12.363313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-125c0d89-f3ec-4382-8f50-505e18affa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:12.374859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 904/1817] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:12 2022] PUT /identity/v3/projects/670e32b4b2a44d0c83d60258d73bdc9e/users/39e0161d99f04f3697e6f3925310ea26/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:12.381091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:12.384270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ed628-cf75-4c97-a2f6-0caf3061966c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:12.384557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ed628-cf75-4c97-a2f6-0caf3061966c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:12.384837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ed628-cf75-4c97-a2f6-0caf3061966c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:12.457062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-af2ed628-cf75-4c97-a2f6-0caf3061966c None None] MFA Rules not processed for user `39e0161d99f04f3697e6f3925310ea26`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:12.457964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-af2ed628-cf75-4c97-a2f6-0caf3061966c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:12.478236 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 914/1818] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:12.485639 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c0d89-f3ec-4382-8f50-505e18affa9a None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:12.487758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-125c0d89-f3ec-4382-8f50-505e18affa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:12.551944 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 905/1819] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:12.556693 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ed628-cf75-4c97-a2f6-0caf3061966c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:12.558100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-af2ed628-cf75-4c97-a2f6-0caf3061966c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:12.585826 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ed628-cf75-4c97-a2f6-0caf3061966c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:12.597600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74451b3a-e53f-4568-8f01-7f007f3406fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:12.598245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74451b3a-e53f-4568-8f01-7f007f3406fa admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:12.598530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74451b3a-e53f-4568-8f01-7f007f3406fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:12.598805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74451b3a-e53f-4568-8f01-7f007f3406fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:12.599445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74451b3a-e53f-4568-8f01-7f007f3406fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:12.601386 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74451b3a-e53f-4568-8f01-7f007f3406fa admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:12.675228 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c0d89-f3ec-4382-8f50-505e18affa9a None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:12.678740 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 906/1820] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:12 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:12.679719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74451b3a-e53f-4568-8f01-7f007f3406fa admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:12.773852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:12.810580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653996e1-9db7-4941-90a8-36a3d702f745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:12.810580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653996e1-9db7-4941-90a8-36a3d702f745 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:12.810580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653996e1-9db7-4941-90a8-36a3d702f745 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:12.810580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653996e1-9db7-4941-90a8-36a3d702f745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:12.828652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653996e1-9db7-4941-90a8-36a3d702f745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:12.830327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653996e1-9db7-4941-90a8-36a3d702f745 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:12.833977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74451b3a-e53f-4568-8f01-7f007f3406fa admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:12.842746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d1454-0f08-4675-966f-8584ad2f64d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:12.843101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 915/1821] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:12.844108 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d1454-0f08-4675-966f-8584ad2f64d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:12.844396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d1454-0f08-4675-966f-8584ad2f64d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:12.844708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d1454-0f08-4675-966f-8584ad2f64d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:12.847401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d1454-0f08-4675-966f-8584ad2f64d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:12.849636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d1454-0f08-4675-966f-8584ad2f64d3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:12.878304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 907/1822] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:12 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:12.882448 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653996e1-9db7-4941-90a8-36a3d702f745 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:12.898366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653996e1-9db7-4941-90a8-36a3d702f745 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:12.906813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be4f3d-92c9-49d4-a19d-77b1ff240d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:12.907941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be4f3d-92c9-49d4-a19d-77b1ff240d05 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:12.908234 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be4f3d-92c9-49d4-a19d-77b1ff240d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:12.908503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be4f3d-92c9-49d4-a19d-77b1ff240d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:12.909917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be4f3d-92c9-49d4-a19d-77b1ff240d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:12.912288 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be4f3d-92c9-49d4-a19d-77b1ff240d05 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:12.930312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d1454-0f08-4675-966f-8584ad2f64d3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:12.931033 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 916/1823] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:12.945756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d1454-0f08-4675-966f-8584ad2f64d3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:12.960417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708d792-fc8c-47ec-bba8-a7e52d2053cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:12.961348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708d792-fc8c-47ec-bba8-a7e52d2053cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:12.961776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708d792-fc8c-47ec-bba8-a7e52d2053cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:12.962146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708d792-fc8c-47ec-bba8-a7e52d2053cf admin admin] PATH_INFO: `/v3/projects/662a6e50bec5432f92461adf17d6252b/users/e6df75c1ec41456c809638011ea4ba19/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:12.995743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c708d792-fc8c-47ec-bba8-a7e52d2053cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=662a6e50bec5432f92461adf17d6252b, user_id=e6df75c1ec41456c809638011ea4ba19, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:12.999675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c708d792-fc8c-47ec-bba8-a7e52d2053cf admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:13.032069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 908/1824] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:12 2022] PUT /identity/v3/projects/662a6e50bec5432f92461adf17d6252b/users/e6df75c1ec41456c809638011ea4ba19/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:13.036134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be4f3d-92c9-49d4-a19d-77b1ff240d05 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:13.046908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be4f3d-92c9-49d4-a19d-77b1ff240d05 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:13.125213 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:13.137064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80a2e4-27d4-4393-9519-ed40e9d1bf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:13.137701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80a2e4-27d4-4393-9519-ed40e9d1bf2e admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:13.138113 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80a2e4-27d4-4393-9519-ed40e9d1bf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:13.138384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80a2e4-27d4-4393-9519-ed40e9d1bf2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:13.139064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80a2e4-27d4-4393-9519-ed40e9d1bf2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:13.141181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80a2e4-27d4-4393-9519-ed40e9d1bf2e admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:13.165455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 917/1825] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:13.169549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708d792-fc8c-47ec-bba8-a7e52d2053cf admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:13.184043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708d792-fc8c-47ec-bba8-a7e52d2053cf admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:13.189372 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b11ef-d76f-4e91-973e-9c1f97870807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:13.190040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b11ef-d76f-4e91-973e-9c1f97870807 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:13.190320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b11ef-d76f-4e91-973e-9c1f97870807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:13.191096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b11ef-d76f-4e91-973e-9c1f97870807 admin admin] PATH_INFO: `/v3/projects/662a6e50bec5432f92461adf17d6252b/users/e6df75c1ec41456c809638011ea4ba19/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:13.193091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b11ef-d76f-4e91-973e-9c1f97870807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=662a6e50bec5432f92461adf17d6252b, user_id=e6df75c1ec41456c809638011ea4ba19, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:13.196181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b11ef-d76f-4e91-973e-9c1f97870807 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:13.207405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 909/1826] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:13 2022] PUT /identity/v3/projects/662a6e50bec5432f92461adf17d6252b/users/e6df75c1ec41456c809638011ea4ba19/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:13.219050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599e1fa-5025-48f5-b976-c3aefa36f798 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:13.219282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599e1fa-5025-48f5-b976-c3aefa36f798 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:13.219556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599e1fa-5025-48f5-b976-c3aefa36f798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:13.248128 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b11ef-d76f-4e91-973e-9c1f97870807 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:13.250861 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cf9b11ef-d76f-4e91-973e-9c1f97870807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:13.269387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-a599e1fa-5025-48f5-b976-c3aefa36f798 None None] MFA Rules not processed for user `e6df75c1ec41456c809638011ea4ba19`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:13.270756 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a599e1fa-5025-48f5-b976-c3aefa36f798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:13.342232 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b11ef-d76f-4e91-973e-9c1f97870807 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:13.347092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e84e1-51cb-4691-803d-c9f2c76b927b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:13.347872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e84e1-51cb-4691-803d-c9f2c76b927b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:13.348150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e84e1-51cb-4691-803d-c9f2c76b927b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:13.348419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e84e1-51cb-4691-803d-c9f2c76b927b admin admin] PATH_INFO: `/v3/projects/6dff9723a7e144268b84c20a815183ef` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:13.349819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e84e1-51cb-4691-803d-c9f2c76b927b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dff9723a7e144268b84c20a815183ef)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:13.351758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e84e1-51cb-4691-803d-c9f2c76b927b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:13.367713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 918/1827] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:13.379239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599e1fa-5025-48f5-b976-c3aefa36f798 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:13.381667 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a599e1fa-5025-48f5-b976-c3aefa36f798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:13.428424 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1e1e84e1-51cb-4691-803d-c9f2c76b927b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dff9723a7e144268b84c20a815183ef', 'request_id': 'req-1e1e84e1-51cb-4691-803d-c9f2c76b927b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:13.436220 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1e1e84e1-51cb-4691-803d-c9f2c76b927b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dff9723a7e144268b84c20a815183ef', 'request_id': 'req-1e1e84e1-51cb-4691-803d-c9f2c76b927b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:13.457103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1e1e84e1-51cb-4691-803d-c9f2c76b927b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dff9723a7e144268b84c20a815183ef', 'request_id': 'req-1e1e84e1-51cb-4691-803d-c9f2c76b927b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:13.473038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599e1fa-5025-48f5-b976-c3aefa36f798 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:13.480572 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1e1e84e1-51cb-4691-803d-c9f2c76b927b admin admin] The token cache is being invalidate because project 6dff9723a7e144268b84c20a815183ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:13.480864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1e1e84e1-51cb-4691-803d-c9f2c76b927b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:13.481528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1e1e84e1-51cb-4691-803d-c9f2c76b927b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:13.483735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 910/1828] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:13 2022] DELETE /identity/v3/projects/6dff9723a7e144268b84c20a815183ef => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:13.488721 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e0e57-cce4-4fb9-a69d-3913805dbd7a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:13.489024 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e0e57-cce4-4fb9-a69d-3913805dbd7a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:13.489285 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e0e57-cce4-4fb9-a69d-3913805dbd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:13.520004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a64e0e57-cce4-4fb9-a69d-3913805dbd7a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:13.521230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a64e0e57-cce4-4fb9-a69d-3913805dbd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:13.531074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:13.542950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2884ed-b871-4608-8dbd-3a7c04a2b595 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:13.545111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2884ed-b871-4608-8dbd-3a7c04a2b595 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:13.547250 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2884ed-b871-4608-8dbd-3a7c04a2b595 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:13.547955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2884ed-b871-4608-8dbd-3a7c04a2b595 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:13.550555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-9b2884ed-b871-4608-8dbd-3a7c04a2b595 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:13.624979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 911/1829] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:13.627252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eb120-68cd-45bb-aad4-d66c5e5749e5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:13.627615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eb120-68cd-45bb-aad4-d66c5e5749e5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:13.627954 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eb120-68cd-45bb-aad4-d66c5e5749e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:13.631708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:13.651028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2884ed-b871-4608-8dbd-3a7c04a2b595 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:13.653482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2884ed-b871-4608-8dbd-3a7c04a2b595 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:13.656196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e45eb120-68cd-45bb-aad4-d66c5e5749e5 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:13.657316 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e45eb120-68cd-45bb-aad4-d66c5e5749e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:13.666215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 919/1830] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:13.670560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 912/1831] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:13.672407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2884ed-b871-4608-8dbd-3a7c04a2b595 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:13.685787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2884ed-b871-4608-8dbd-3a7c04a2b595 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:13.692876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663bf15-784f-4eaf-9ac0-a9ab043355b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:13.695267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663bf15-784f-4eaf-9ac0-a9ab043355b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:13.695555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663bf15-784f-4eaf-9ac0-a9ab043355b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:13.695824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663bf15-784f-4eaf-9ac0-a9ab043355b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:13.696515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663bf15-784f-4eaf-9ac0-a9ab043355b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:13.696809 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663bf15-784f-4eaf-9ac0-a9ab043355b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:13.698542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663bf15-784f-4eaf-9ac0-a9ab043355b9 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:13.717311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45eb120-68cd-45bb-aad4-d66c5e5749e5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:13.718933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 920/1832] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:32:13.727914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45eb120-68cd-45bb-aad4-d66c5e5749e5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:13.734641 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55303297-cfed-4f65-b574-b6f8f84c4edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:13.735295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55303297-cfed-4f65-b574-b6f8f84c4edd admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:13.735724 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55303297-cfed-4f65-b574-b6f8f84c4edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:13.736001 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55303297-cfed-4f65-b574-b6f8f84c4edd admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:13.736672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55303297-cfed-4f65-b574-b6f8f84c4edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:13.739105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55303297-cfed-4f65-b574-b6f8f84c4edd admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:13.766420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663bf15-784f-4eaf-9ac0-a9ab043355b9 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:13.770393 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a663bf15-784f-4eaf-9ac0-a9ab043355b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:13.784276 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 913/1833] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:13 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:13.784485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663bf15-784f-4eaf-9ac0-a9ab043355b9 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:13.786317 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55303297-cfed-4f65-b574-b6f8f84c4edd admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:13.859290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d95467-1066-42eb-be50-d97f8fbf2565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:13.859946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d95467-1066-42eb-be50-d97f8fbf2565 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:13.860225 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d95467-1066-42eb-be50-d97f8fbf2565 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:13.860499 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d95467-1066-42eb-be50-d97f8fbf2565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:13.862372 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e9d95467-1066-42eb-be50-d97f8fbf2565 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:13.863464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55303297-cfed-4f65-b574-b6f8f84c4edd admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:13.868696 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7738bc8-9731-462c-9995-b25ce6ae34b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:13.869537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7738bc8-9731-462c-9995-b25ce6ae34b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:13.869646 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7738bc8-9731-462c-9995-b25ce6ae34b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:13.869871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7738bc8-9731-462c-9995-b25ce6ae34b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:13.870560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7738bc8-9731-462c-9995-b25ce6ae34b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:13.872502 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7738bc8-9731-462c-9995-b25ce6ae34b7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:13.903381 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 914/1834] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:13 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:13.904611 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7738bc8-9731-462c-9995-b25ce6ae34b7 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:13.914701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7738bc8-9731-462c-9995-b25ce6ae34b7 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:13.919825 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09d851-7fa6-47a6-a149-49b412f3a4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:13.920534 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09d851-7fa6-47a6-a149-49b412f3a4ac admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:13.920806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09d851-7fa6-47a6-a149-49b412f3a4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:13.921072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09d851-7fa6-47a6-a149-49b412f3a4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:13.921734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09d851-7fa6-47a6-a149-49b412f3a4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:13.923411 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09d851-7fa6-47a6-a149-49b412f3a4ac admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:13.935001 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 915/1835] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:32:13.937016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d95467-1066-42eb-be50-d97f8fbf2565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:13.938583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d95467-1066-42eb-be50-d97f8fbf2565 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:13.938823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09d851-7fa6-47a6-a149-49b412f3a4ac admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:13.949034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 921/1836] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:13.949370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09d851-7fa6-47a6-a149-49b412f3a4ac admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:13.954483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc9a33-9790-4720-b98c-05e1f7382de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:13.955141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc9a33-9790-4720-b98c-05e1f7382de7 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:13.955430 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc9a33-9790-4720-b98c-05e1f7382de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:13.955706 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc9a33-9790-4720-b98c-05e1f7382de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:13.956354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc9a33-9790-4720-b98c-05e1f7382de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:13.958015 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc9a33-9790-4720-b98c-05e1f7382de7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:13.970787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 916/1837] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:32:13.975520 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d95467-1066-42eb-be50-d97f8fbf2565 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:13.998833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d95467-1066-42eb-be50-d97f8fbf2565 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:14.006631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30032b8-3bdc-4ca1-b68b-043fbf5b1b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:14.007182 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30032b8-3bdc-4ca1-b68b-043fbf5b1b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:14.007766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30032b8-3bdc-4ca1-b68b-043fbf5b1b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:14.007885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30032b8-3bdc-4ca1-b68b-043fbf5b1b93 admin admin] PATH_INFO: `/v3/projects/dbc3c87ab6534e8b86bb9b827416f8fb/users/90a3e5b850ac4b098d3ca5121bb02c14/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:14.029219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30032b8-3bdc-4ca1-b68b-043fbf5b1b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbc3c87ab6534e8b86bb9b827416f8fb, user_id=90a3e5b850ac4b098d3ca5121bb02c14, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:14.032012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30032b8-3bdc-4ca1-b68b-043fbf5b1b93 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:14.051215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 922/1838] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:13 2022] PUT /identity/v3/projects/dbc3c87ab6534e8b86bb9b827416f8fb/users/90a3e5b850ac4b098d3ca5121bb02c14/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:14.076394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066bf41-f50b-4cb4-83f3-ef81ac9f9e76 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:14.076678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066bf41-f50b-4cb4-83f3-ef81ac9f9e76 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:14.076953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066bf41-f50b-4cb4-83f3-ef81ac9f9e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:14.109650 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4066bf41-f50b-4cb4-83f3-ef81ac9f9e76 None None] MFA Rules not processed for user `90a3e5b850ac4b098d3ca5121bb02c14`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:14.111630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4066bf41-f50b-4cb4-83f3-ef81ac9f9e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:14.173445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:14.198897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 917/1839] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:14.210828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30032b8-3bdc-4ca1-b68b-043fbf5b1b93 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:14.267979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30032b8-3bdc-4ca1-b68b-043fbf5b1b93 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:14.271478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04b975-ac43-4629-ba07-c76a19c9df35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:14.272100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04b975-ac43-4629-ba07-c76a19c9df35 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:14.272373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04b975-ac43-4629-ba07-c76a19c9df35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:14.272638 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04b975-ac43-4629-ba07-c76a19c9df35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:14.281111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04b975-ac43-4629-ba07-c76a19c9df35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:14.282558 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04b975-ac43-4629-ba07-c76a19c9df35 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:14.291762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 923/1840] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:16.213283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066bf41-f50b-4cb4-83f3-ef81ac9f9e76 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:16.214928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4066bf41-f50b-4cb4-83f3-ef81ac9f9e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:16.318326 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066bf41-f50b-4cb4-83f3-ef81ac9f9e76 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:16.325370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb9368-2371-47db-b591-c22d7c129ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:16.326343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb9368-2371-47db-b591-c22d7c129ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:16.326927 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb9368-2371-47db-b591-c22d7c129ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:16.327693 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb9368-2371-47db-b591-c22d7c129ad7 admin admin] PATH_INFO: `/v3/users/67dc89a481ff44ea86f4719de8186b72` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:16.329586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb9368-2371-47db-b591-c22d7c129ad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67dc89a481ff44ea86f4719de8186b72)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:16.331995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb9368-2371-47db-b591-c22d7c129ad7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:16.419959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e8fb9368-2371-47db-b591-c22d7c129ad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67dc89a481ff44ea86f4719de8186b72'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:16.420889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e8fb9368-2371-47db-b591-c22d7c129ad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67dc89a481ff44ea86f4719de8186b72'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:16.421738 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e8fb9368-2371-47db-b591-c22d7c129ad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67dc89a481ff44ea86f4719de8186b72'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:16.431061 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e8fb9368-2371-47db-b591-c22d7c129ad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67dc89a481ff44ea86f4719de8186b72'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:16.445484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e8fb9368-2371-47db-b591-c22d7c129ad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67dc89a481ff44ea86f4719de8186b72'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:16.464896 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 918/1841] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:16 2022] DELETE /identity/v3/users/67dc89a481ff44ea86f4719de8186b72 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:16.473110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6f602-d8e3-4a10-b07d-b5bcb302905e None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:16.473110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6f602-d8e3-4a10-b07d-b5bcb302905e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:16.473110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6f602-d8e3-4a10-b07d-b5bcb302905e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:16.494879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:16.509023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-fce6f602-d8e3-4a10-b07d-b5bcb302905e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:16.511645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-fce6f602-d8e3-4a10-b07d-b5bcb302905e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:16.621512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 924/1842] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:16.632940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb9368-2371-47db-b591-c22d7c129ad7 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:16.638321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e8fb9368-2371-47db-b591-c22d7c129ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:16.744464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb9368-2371-47db-b591-c22d7c129ad7 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:16.750020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d9834-2028-4931-8405-9bd5ebde9a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:16.750684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d9834-2028-4931-8405-9bd5ebde9a32 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:16.751346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d9834-2028-4931-8405-9bd5ebde9a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:16.751346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d9834-2028-4931-8405-9bd5ebde9a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:16.773767 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d9834-2028-4931-8405-9bd5ebde9a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:16.775819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d9834-2028-4931-8405-9bd5ebde9a32 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:16.790283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 919/1843] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:17.579412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6f602-d8e3-4a10-b07d-b5bcb302905e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:17.581252 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-fce6f602-d8e3-4a10-b07d-b5bcb302905e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:17.623396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6f602-d8e3-4a10-b07d-b5bcb302905e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:17.629498 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:17.630162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:17.630525 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:17.630798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] PATH_INFO: `/v3/projects/ea8147c9ea604f3783001b054aee6786` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:17.631967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea8147c9ea604f3783001b054aee6786)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:17.634304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:17.714920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea8147c9ea604f3783001b054aee6786', 'request_id': 'req-8c553536-1b61-4430-9b43-80743b7ccaf9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:17.726960 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea8147c9ea604f3783001b054aee6786', 'request_id': 'req-8c553536-1b61-4430-9b43-80743b7ccaf9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:17.749324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea8147c9ea604f3783001b054aee6786', 'request_id': 'req-8c553536-1b61-4430-9b43-80743b7ccaf9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:17.756691 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d9834-2028-4931-8405-9bd5ebde9a32 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:17.758996 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-438d9834-2028-4931-8405-9bd5ebde9a32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:17.765474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] The token cache is being invalidate because project ea8147c9ea604f3783001b054aee6786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:17.765767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:17.766435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:17.768347 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 925/1844] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:17 2022] DELETE /identity/v3/projects/ea8147c9ea604f3783001b054aee6786 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:17.774178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:17.775687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:17.784314 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:17.787837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d9834-2028-4931-8405-9bd5ebde9a32 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:17.943584 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385bc005-d0a9-49fd-9c61-1eac3c4e2a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:17.944263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bc005-d0a9-49fd-9c61-1eac3c4e2a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:17.944543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bc005-d0a9-49fd-9c61-1eac3c4e2a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:17.944854 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bc005-d0a9-49fd-9c61-1eac3c4e2a10 admin admin] PATH_INFO: `/v3/users/90a3e5b850ac4b098d3ca5121bb02c14` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:17.947784 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385bc005-d0a9-49fd-9c61-1eac3c4e2a10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90a3e5b850ac4b098d3ca5121bb02c14)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:17.952781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385bc005-d0a9-49fd-9c61-1eac3c4e2a10 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:17.981004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c553536-1b61-4430-9b43-80743b7ccaf9 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:18.006168 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdcaa3-b0d5-4faa-a660-6c3752ab9f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:18.007833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdcaa3-b0d5-4faa-a660-6c3752ab9f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:18.008250 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdcaa3-b0d5-4faa-a660-6c3752ab9f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:18.008528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdcaa3-b0d5-4faa-a660-6c3752ab9f5b admin admin] PATH_INFO: `/v3/users/b83038a71c4542138a72a608507cb51d` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:18.010152 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fdcaa3-b0d5-4faa-a660-6c3752ab9f5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b83038a71c4542138a72a608507cb51d)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:18.021092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fdcaa3-b0d5-4faa-a660-6c3752ab9f5b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:18.115918 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:18.158232 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-29fdcaa3-b0d5-4faa-a660-6c3752ab9f5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b83038a71c4542138a72a608507cb51d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:18.160856 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-29fdcaa3-b0d5-4faa-a660-6c3752ab9f5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b83038a71c4542138a72a608507cb51d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:18.166905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-385bc005-d0a9-49fd-9c61-1eac3c4e2a10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90a3e5b850ac4b098d3ca5121bb02c14'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:18.167128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-29fdcaa3-b0d5-4faa-a660-6c3752ab9f5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b83038a71c4542138a72a608507cb51d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:18.169807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-385bc005-d0a9-49fd-9c61-1eac3c4e2a10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90a3e5b850ac4b098d3ca5121bb02c14'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:18.174693 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-385bc005-d0a9-49fd-9c61-1eac3c4e2a10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90a3e5b850ac4b098d3ca5121bb02c14'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:18.187770 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-29fdcaa3-b0d5-4faa-a660-6c3752ab9f5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b83038a71c4542138a72a608507cb51d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:18.197662 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-385bc005-d0a9-49fd-9c61-1eac3c4e2a10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90a3e5b850ac4b098d3ca5121bb02c14'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:18.243098 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-29fdcaa3-b0d5-4faa-a660-6c3752ab9f5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b83038a71c4542138a72a608507cb51d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:18.249447 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-385bc005-d0a9-49fd-9c61-1eac3c4e2a10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90a3e5b850ac4b098d3ca5121bb02c14'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:18.287419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 920/1846] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:17 2022] DELETE /identity/v3/users/90a3e5b850ac4b098d3ca5121bb02c14 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:18.287650 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 926/1846] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:17 2022] DELETE /identity/v3/users/b83038a71c4542138a72a608507cb51d => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:18.287765 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372e94e-8675-480c-b67c-a6fedb9f525f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:18.289121 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f5f20-f9a2-4453-84a1-7361d1e518e7 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:18.289399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f5f20-f9a2-4453-84a1-7361d1e518e7 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:18.289910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372e94e-8675-480c-b67c-a6fedb9f525f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:18.290123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f5f20-f9a2-4453-84a1-7361d1e518e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:18.290212 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372e94e-8675-480c-b67c-a6fedb9f525f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:18.330031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0372e94e-8675-480c-b67c-a6fedb9f525f None None] MFA Rules not processed for user `aa65b6854dc54e289cc3e8e0b3746dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:18.332159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0372e94e-8675-480c-b67c-a6fedb9f525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:18.355452 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f03f5f20-f9a2-4453-84a1-7361d1e518e7 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:18.356702 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f03f5f20-f9a2-4453-84a1-7361d1e518e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:18.460448 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 927/1847] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:18.499201 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 921/1848] 10.222.0.25 () {58 vars in 1015 bytes} [Fri Sep 9 16:32:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:18.512726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f5f20-f9a2-4453-84a1-7361d1e518e7 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:18.514408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f03f5f20-f9a2-4453-84a1-7361d1e518e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:18.537350 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f5f20-f9a2-4453-84a1-7361d1e518e7 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:18.549652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80aeeb6-8aa6-4553-a163-d505520a6c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:18.550301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aeeb6-8aa6-4553-a163-d505520a6c7d service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:18.550615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aeeb6-8aa6-4553-a163-d505520a6c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:18.550850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aeeb6-8aa6-4553-a163-d505520a6c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:18.577686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80aeeb6-8aa6-4553-a163-d505520a6c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:18.579543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80aeeb6-8aa6-4553-a163-d505520a6c7d service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:18.594037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 928/1849] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:19.154360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372e94e-8675-480c-b67c-a6fedb9f525f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:19.157251 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0372e94e-8675-480c-b67c-a6fedb9f525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:19.194880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372e94e-8675-480c-b67c-a6fedb9f525f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:19.201860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:19.202779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:19.203958 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:19.204439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d admin admin] PATH_INFO: `/v3/projects/d77a194dc2f54e8d8843f361f2088266` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:19.206237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d77a194dc2f54e8d8843f361f2088266)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:19.209082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:19.290825 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:19.338642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd77a194dc2f54e8d8843f361f2088266', 'request_id': 'req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:19.349284 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd77a194dc2f54e8d8843f361f2088266', 'request_id': 'req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:19.351214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80aeeb6-8aa6-4553-a163-d505520a6c7d service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:19.352892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e80aeeb6-8aa6-4553-a163-d505520a6c7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:19.370679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd77a194dc2f54e8d8843f361f2088266', 'request_id': 'req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:19.377558 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80aeeb6-8aa6-4553-a163-d505520a6c7d service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:19.391703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d admin admin] The token cache is being invalidate because project d77a194dc2f54e8d8843f361f2088266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:19.392370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:19.393217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8f4e3d3d-85fe-4146-b045-68424aaa1d8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:19.395252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 922/1850] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:19 2022] DELETE /identity/v3/projects/d77a194dc2f54e8d8843f361f2088266 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:19.399888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cb19c-15ea-4578-965c-1f2f9d0ae9a8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:19.400168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cb19c-15ea-4578-965c-1f2f9d0ae9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:19.400438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cb19c-15ea-4578-965c-1f2f9d0ae9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:19.447225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-944cb19c-15ea-4578-965c-1f2f9d0ae9a8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:19.448654 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-944cb19c-15ea-4578-965c-1f2f9d0ae9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:19.478459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fc25a-3199-4512-a53c-07979386c50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:19.479455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fc25a-3199-4512-a53c-07979386c50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:19.479963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fc25a-3199-4512-a53c-07979386c50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:19.480326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fc25a-3199-4512-a53c-07979386c50c admin admin] PATH_INFO: `/v3/projects/dbc3c87ab6534e8b86bb9b827416f8fb` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:19.481731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600fc25a-3199-4512-a53c-07979386c50c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbc3c87ab6534e8b86bb9b827416f8fb)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:19.483973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600fc25a-3199-4512-a53c-07979386c50c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:19.559739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:19.561049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-600fc25a-3199-4512-a53c-07979386c50c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbc3c87ab6534e8b86bb9b827416f8fb', 'request_id': 'req-600fc25a-3199-4512-a53c-07979386c50c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:19.569620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-600fc25a-3199-4512-a53c-07979386c50c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbc3c87ab6534e8b86bb9b827416f8fb', 'request_id': 'req-600fc25a-3199-4512-a53c-07979386c50c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:19.593290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-600fc25a-3199-4512-a53c-07979386c50c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbc3c87ab6534e8b86bb9b827416f8fb', 'request_id': 'req-600fc25a-3199-4512-a53c-07979386c50c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:19.607592 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-600fc25a-3199-4512-a53c-07979386c50c admin admin] The token cache is being invalidate because project dbc3c87ab6534e8b86bb9b827416f8fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:19.607946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-600fc25a-3199-4512-a53c-07979386c50c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:19.608583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-600fc25a-3199-4512-a53c-07979386c50c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:19.619117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 923/1851] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:19.619321 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 929/1852] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:19 2022] DELETE /identity/v3/projects/dbc3c87ab6534e8b86bb9b827416f8fb => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:19.620545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f58d5-6ae2-46f0-8109-8eed12fa441a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:19.620838 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f58d5-6ae2-46f0-8109-8eed12fa441a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:19.621113 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f58d5-6ae2-46f0-8109-8eed12fa441a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:19.624112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608b2f4-d06b-41a0-80bd-6c166ff43270 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:19.624416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608b2f4-d06b-41a0-80bd-6c166ff43270 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:19.624771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608b2f4-d06b-41a0-80bd-6c166ff43270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:19.642105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5d1f58d5-6ae2-46f0-8109-8eed12fa441a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:19.643465 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5d1f58d5-6ae2-46f0-8109-8eed12fa441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:19.653498 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-0608b2f4-d06b-41a0-80bd-6c166ff43270 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:19.654767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0608b2f4-d06b-41a0-80bd-6c166ff43270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:19.718195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 924/1853] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:19.726136 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d69448-a877-4a90-b1bb-47c431f1fae8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:19.726428 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d69448-a877-4a90-b1bb-47c431f1fae8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:19.726737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d69448-a877-4a90-b1bb-47c431f1fae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:19.745420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 930/1854] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:19.748182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b3d69448-a877-4a90-b1bb-47c431f1fae8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:19.749585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b3d69448-a877-4a90-b1bb-47c431f1fae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:19.750222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608b2f4-d06b-41a0-80bd-6c166ff43270 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:19.762643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 925/1855] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:19.763428 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608b2f4-d06b-41a0-80bd-6c166ff43270 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:19.764569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d69448-a877-4a90-b1bb-47c431f1fae8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:19.770103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91173f5-bb54-4b21-9de3-dce0a8a41f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:19.770916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91173f5-bb54-4b21-9de3-dce0a8a41f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:19.771274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91173f5-bb54-4b21-9de3-dce0a8a41f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:19.771845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91173f5-bb54-4b21-9de3-dce0a8a41f6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:19.772521 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91173f5-bb54-4b21-9de3-dce0a8a41f6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:19.772824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91173f5-bb54-4b21-9de3-dce0a8a41f6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:19.774611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91173f5-bb54-4b21-9de3-dce0a8a41f6b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:19.775214 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d69448-a877-4a90-b1bb-47c431f1fae8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:19.790028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ae49f-40b8-49b8-98dc-703f2ccf8202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:19.791283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ae49f-40b8-49b8-98dc-703f2ccf8202 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:19.791567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ae49f-40b8-49b8-98dc-703f2ccf8202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:19.791839 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ae49f-40b8-49b8-98dc-703f2ccf8202 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:19.792562 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ae49f-40b8-49b8-98dc-703f2ccf8202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:19.792844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ae49f-40b8-49b8-98dc-703f2ccf8202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:19.795467 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ae49f-40b8-49b8-98dc-703f2ccf8202 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:19.805073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 931/1856] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:32:19.806188 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91173f5-bb54-4b21-9de3-dce0a8a41f6b admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:19.811881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 926/1857] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:32:19.821122 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ae49f-40b8-49b8-98dc-703f2ccf8202 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:19.826053 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91173f5-bb54-4b21-9de3-dce0a8a41f6b admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:19.833224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9f59e-46b0-48c8-aee8-bfc74eb290be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:19.833937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9f59e-46b0-48c8-aee8-bfc74eb290be admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:19.834215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9f59e-46b0-48c8-aee8-bfc74eb290be admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:19.834954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9f59e-46b0-48c8-aee8-bfc74eb290be admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:19.835723 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9f59e-46b0-48c8-aee8-bfc74eb290be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:19.838140 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9f59e-46b0-48c8-aee8-bfc74eb290be admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:19.839897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ae49f-40b8-49b8-98dc-703f2ccf8202 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:19.847503 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafab1f9-c0b8-432d-9cb7-cdd8783af9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:19.848216 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafab1f9-c0b8-432d-9cb7-cdd8783af9f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:19.848563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafab1f9-c0b8-432d-9cb7-cdd8783af9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:19.859767 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafab1f9-c0b8-432d-9cb7-cdd8783af9f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:19.860673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafab1f9-c0b8-432d-9cb7-cdd8783af9f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:19.862668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafab1f9-c0b8-432d-9cb7-cdd8783af9f3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:19.890898 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 932/1858] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:19 2022] POST /identity/v3/projects => generated 375 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:19.897087 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9f59e-46b0-48c8-aee8-bfc74eb290be admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:19.905324 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 927/1859] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:19 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:19.906621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafab1f9-c0b8-432d-9cb7-cdd8783af9f3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:19.933492 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:20.051350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafab1f9-c0b8-432d-9cb7-cdd8783af9f3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:20.070808 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d1fa4-85a7-4dfa-b8d4-5b3a4b412dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:20.071461 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d1fa4-85a7-4dfa-b8d4-5b3a4b412dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:20.071735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d1fa4-85a7-4dfa-b8d4-5b3a4b412dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:20.072003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d1fa4-85a7-4dfa-b8d4-5b3a4b412dd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:20.072638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d1fa4-85a7-4dfa-b8d4-5b3a4b412dd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:20.096136 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d1fa4-85a7-4dfa-b8d4-5b3a4b412dd8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:20.105792 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9f59e-46b0-48c8-aee8-bfc74eb290be admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:20.115819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734eec93-2dce-453c-93c9-c070216bb56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:20.117510 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eec93-2dce-453c-93c9-c070216bb56a admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:20.117877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eec93-2dce-453c-93c9-c070216bb56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:20.118700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eec93-2dce-453c-93c9-c070216bb56a admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:20.120056 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734eec93-2dce-453c-93c9-c070216bb56a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:20.132058 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734eec93-2dce-453c-93c9-c070216bb56a admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:20.153726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 928/1860] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:19 2022] POST /identity/v3/users => generated 292 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:20.161016 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d1fa4-85a7-4dfa-b8d4-5b3a4b412dd8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:20.187709 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 933/1861] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:19 2022] POST /identity/v3/users => generated 288 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:20.192343 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734eec93-2dce-453c-93c9-c070216bb56a admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:20.197679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d1fa4-85a7-4dfa-b8d4-5b3a4b412dd8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:20.212499 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d150f66-f4ac-4030-b468-f57c4864b070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:20.213532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d150f66-f4ac-4030-b468-f57c4864b070 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:20.215891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d150f66-f4ac-4030-b468-f57c4864b070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:20.216815 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734eec93-2dce-453c-93c9-c070216bb56a admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:20.220706 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d150f66-f4ac-4030-b468-f57c4864b070 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:20.221480 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d150f66-f4ac-4030-b468-f57c4864b070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:20.223205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d150f66-f4ac-4030-b468-f57c4864b070 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:20.227046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6aa1a-b937-4155-a1cd-c55fb6fdba64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:20.227686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6aa1a-b937-4155-a1cd-c55fb6fdba64 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:20.227963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6aa1a-b937-4155-a1cd-c55fb6fdba64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:20.229017 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6aa1a-b937-4155-a1cd-c55fb6fdba64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:20.229953 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6aa1a-b937-4155-a1cd-c55fb6fdba64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:20.231635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6aa1a-b937-4155-a1cd-c55fb6fdba64 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:20.251212 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 934/1862] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:20.264067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6aa1a-b937-4155-a1cd-c55fb6fdba64 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:20.269703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 929/1863] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:20.273565 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d150f66-f4ac-4030-b468-f57c4864b070 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:20.278420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6aa1a-b937-4155-a1cd-c55fb6fdba64 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:20.284674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb46be-3edd-4e77-a911-6fd4704b67fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:20.285606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb46be-3edd-4e77-a911-6fd4704b67fa admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:20.286138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb46be-3edd-4e77-a911-6fd4704b67fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:20.287550 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb46be-3edd-4e77-a911-6fd4704b67fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:20.287664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb46be-3edd-4e77-a911-6fd4704b67fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:20.289402 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb46be-3edd-4e77-a911-6fd4704b67fa admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:20.294920 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d150f66-f4ac-4030-b468-f57c4864b070 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:20.296794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:20.308385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01a11d-4a2d-4e0f-82cd-99a73fe2ab49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:20.309101 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01a11d-4a2d-4e0f-82cd-99a73fe2ab49 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:20.311379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01a11d-4a2d-4e0f-82cd-99a73fe2ab49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:20.311683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01a11d-4a2d-4e0f-82cd-99a73fe2ab49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:20.314144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01a11d-4a2d-4e0f-82cd-99a73fe2ab49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:20.320292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01a11d-4a2d-4e0f-82cd-99a73fe2ab49 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:20.346690 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 935/1864] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:32:20.350704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb46be-3edd-4e77-a911-6fd4704b67fa admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:20.356770 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 930/1865] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:20.361086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01a11d-4a2d-4e0f-82cd-99a73fe2ab49 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:20.378241 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01a11d-4a2d-4e0f-82cd-99a73fe2ab49 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:20.379402 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb46be-3edd-4e77-a911-6fd4704b67fa admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:20.390111 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daf7ce-8279-40b2-b4d6-9fb590e57a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:20.391015 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123652d-7b17-4fba-a797-28757fb445e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:20.392107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123652d-7b17-4fba-a797-28757fb445e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:20.392937 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daf7ce-8279-40b2-b4d6-9fb590e57a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:20.393264 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daf7ce-8279-40b2-b4d6-9fb590e57a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:20.393454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123652d-7b17-4fba-a797-28757fb445e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:20.393553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daf7ce-8279-40b2-b4d6-9fb590e57a16 admin admin] PATH_INFO: `/v3/projects/dc185f2afd1246669850674a10e608d0/users/47976541b90846fc9c8b9ce5743bcc4c/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:20.394876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123652d-7b17-4fba-a797-28757fb445e5 admin admin] PATH_INFO: `/v3/projects/caaf15310a3e45a6b8febb4566e72d79/users/b56d909c22244d7d80f6c366019b4029/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:20.432129 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daf7ce-8279-40b2-b4d6-9fb590e57a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc185f2afd1246669850674a10e608d0, user_id=47976541b90846fc9c8b9ce5743bcc4c, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:20.437209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daf7ce-8279-40b2-b4d6-9fb590e57a16 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:20.439109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9123652d-7b17-4fba-a797-28757fb445e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caaf15310a3e45a6b8febb4566e72d79, user_id=b56d909c22244d7d80f6c366019b4029, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:20.444090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9123652d-7b17-4fba-a797-28757fb445e5 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:20.479087 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 931/1866] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:20 2022] PUT /identity/v3/projects/dc185f2afd1246669850674a10e608d0/users/47976541b90846fc9c8b9ce5743bcc4c/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:20.481774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 936/1867] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:20 2022] PUT /identity/v3/projects/caaf15310a3e45a6b8febb4566e72d79/users/b56d909c22244d7d80f6c366019b4029/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:20.493838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7771d-2c12-4b77-863d-dfac07f3e26b None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:20.494123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7771d-2c12-4b77-863d-dfac07f3e26b None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:20.494488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7771d-2c12-4b77-863d-dfac07f3e26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:20.501579 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726697a8-db53-420b-9f0c-89e92c723c6b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:20.501874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726697a8-db53-420b-9f0c-89e92c723c6b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:20.502145 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726697a8-db53-420b-9f0c-89e92c723c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:20.583952 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-726697a8-db53-420b-9f0c-89e92c723c6b None None] MFA Rules not processed for user `47976541b90846fc9c8b9ce5743bcc4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:20.587350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-726697a8-db53-420b-9f0c-89e92c723c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:20.603651 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-6fc7771d-2c12-4b77-863d-dfac07f3e26b None None] MFA Rules not processed for user `b56d909c22244d7d80f6c366019b4029`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:20.607695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-6fc7771d-2c12-4b77-863d-dfac07f3e26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:20.741146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 932/1868] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:20.745320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726697a8-db53-420b-9f0c-89e92c723c6b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:20.760621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 937/1869] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:20.778868 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7771d-2c12-4b77-863d-dfac07f3e26b None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:20.780489 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-6fc7771d-2c12-4b77-863d-dfac07f3e26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:20.801180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:20.816679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726697a8-db53-420b-9f0c-89e92c723c6b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:20.821399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6e3f4-dddc-46bd-85f7-44b5f6a628d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:20.822012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6e3f4-dddc-46bd-85f7-44b5f6a628d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:20.822287 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6e3f4-dddc-46bd-85f7-44b5f6a628d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:20.822582 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6e3f4-dddc-46bd-85f7-44b5f6a628d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:20.823210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6e3f4-dddc-46bd-85f7-44b5f6a628d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:20.825115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6e3f4-dddc-46bd-85f7-44b5f6a628d9 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:20.843390 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7771d-2c12-4b77-863d-dfac07f3e26b None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:20.847435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc17d6-18d9-4b74-9443-70ebd5665512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:20.848080 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc17d6-18d9-4b74-9443-70ebd5665512 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:20.848359 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc17d6-18d9-4b74-9443-70ebd5665512 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:20.848627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc17d6-18d9-4b74-9443-70ebd5665512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:20.858198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 933/1870] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:20 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:32:20.858380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc17d6-18d9-4b74-9443-70ebd5665512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:20.858969 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc17d6-18d9-4b74-9443-70ebd5665512 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:20.859204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6e3f4-dddc-46bd-85f7-44b5f6a628d9 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:20.868154 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 938/1871] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:20.922077 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6e3f4-dddc-46bd-85f7-44b5f6a628d9 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:20.927519 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a5e12-6441-4e2c-9bbf-5a721f2acf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:20.928143 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a5e12-6441-4e2c-9bbf-5a721f2acf0c admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:20.928420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a5e12-6441-4e2c-9bbf-5a721f2acf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:20.928810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a5e12-6441-4e2c-9bbf-5a721f2acf0c admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:20.929431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a5e12-6441-4e2c-9bbf-5a721f2acf0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:20.931603 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a5e12-6441-4e2c-9bbf-5a721f2acf0c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:20.959054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 934/1872] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:20 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:20.960282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc17d6-18d9-4b74-9443-70ebd5665512 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:20.971851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc17d6-18d9-4b74-9443-70ebd5665512 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:20.976620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc3128-35bf-4b82-b69f-3c9e380bbf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:20.977249 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc3128-35bf-4b82-b69f-3c9e380bbf0d admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:20.977528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc3128-35bf-4b82-b69f-3c9e380bbf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:20.977933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc3128-35bf-4b82-b69f-3c9e380bbf0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:20.978732 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc3128-35bf-4b82-b69f-3c9e380bbf0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:20.980491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc3128-35bf-4b82-b69f-3c9e380bbf0d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:20.991266 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 939/1873] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:32:20.995285 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a5e12-6441-4e2c-9bbf-5a721f2acf0c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:21.005381 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a5e12-6441-4e2c-9bbf-5a721f2acf0c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:21.010219 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b949e15-210f-4f19-a3bf-9a07e17bb5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:21.010864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b949e15-210f-4f19-a3bf-9a07e17bb5c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:21.011178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b949e15-210f-4f19-a3bf-9a07e17bb5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:21.011411 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b949e15-210f-4f19-a3bf-9a07e17bb5c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:21.012054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b949e15-210f-4f19-a3bf-9a07e17bb5c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:21.013665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b949e15-210f-4f19-a3bf-9a07e17bb5c7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:21.023794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 935/1874] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:21.028350 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc3128-35bf-4b82-b69f-3c9e380bbf0d admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:21.042342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc3128-35bf-4b82-b69f-3c9e380bbf0d admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:21.046972 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a613de-b8fc-4a0c-83f3-3d7eacaed851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:21.047643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a613de-b8fc-4a0c-83f3-3d7eacaed851 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:21.047920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a613de-b8fc-4a0c-83f3-3d7eacaed851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:21.048191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a613de-b8fc-4a0c-83f3-3d7eacaed851 admin admin] PATH_INFO: `/v3/projects/a341eb72d4664b26a7d1816c00b934ff/users/256fb32309a04a2e9261031ac2a4ac19/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:21.064163 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a613de-b8fc-4a0c-83f3-3d7eacaed851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a341eb72d4664b26a7d1816c00b934ff, user_id=256fb32309a04a2e9261031ac2a4ac19, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:21.066935 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a613de-b8fc-4a0c-83f3-3d7eacaed851 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:21.084909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 940/1875] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:21 2022] PUT /identity/v3/projects/a341eb72d4664b26a7d1816c00b934ff/users/256fb32309a04a2e9261031ac2a4ac19/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:21.096382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44fe45-c43c-4e44-8603-6a02cc80da44 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:21.096667 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44fe45-c43c-4e44-8603-6a02cc80da44 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:21.096937 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44fe45-c43c-4e44-8603-6a02cc80da44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:21.126003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-fa44fe45-c43c-4e44-8603-6a02cc80da44 None None] MFA Rules not processed for user `256fb32309a04a2e9261031ac2a4ac19`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:21.127317 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-fa44fe45-c43c-4e44-8603-6a02cc80da44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:21.199044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 936/1876] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:21.203803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a613de-b8fc-4a0c-83f3-3d7eacaed851 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:21.237043 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:21.265863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a613de-b8fc-4a0c-83f3-3d7eacaed851 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:21.270762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1feef5-3e91-4566-9980-49904ad7d9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:21.271362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1feef5-3e91-4566-9980-49904ad7d9eb admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:21.271635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1feef5-3e91-4566-9980-49904ad7d9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:21.271905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1feef5-3e91-4566-9980-49904ad7d9eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:21.272560 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1feef5-3e91-4566-9980-49904ad7d9eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:21.274596 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1feef5-3e91-4566-9980-49904ad7d9eb admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:21.303021 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 941/1877] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:21 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:21.307207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44fe45-c43c-4e44-8603-6a02cc80da44 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:21.318558 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44fe45-c43c-4e44-8603-6a02cc80da44 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:21.365410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:21.388738 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8f2ad-b34a-4cd1-8da6-fab64ab6d7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:21.389957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f2ad-b34a-4cd1-8da6-fab64ab6d7bf admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:21.390459 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f2ad-b34a-4cd1-8da6-fab64ab6d7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:21.391051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f2ad-b34a-4cd1-8da6-fab64ab6d7bf admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:21.391902 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8f2ad-b34a-4cd1-8da6-fab64ab6d7bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:21.395153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8f2ad-b34a-4cd1-8da6-fab64ab6d7bf admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:21.421391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 937/1878] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:21 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:21.422856 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1feef5-3e91-4566-9980-49904ad7d9eb admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:21.433685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1feef5-3e91-4566-9980-49904ad7d9eb admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:21.438799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f32c0-7e01-4197-a71e-cbb7e3262960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:21.439422 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f32c0-7e01-4197-a71e-cbb7e3262960 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:21.439697 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f32c0-7e01-4197-a71e-cbb7e3262960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:21.439962 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f32c0-7e01-4197-a71e-cbb7e3262960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:21.440625 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f32c0-7e01-4197-a71e-cbb7e3262960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:21.442198 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f32c0-7e01-4197-a71e-cbb7e3262960 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:21.453279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 942/1879] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:21.457393 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8f2ad-b34a-4cd1-8da6-fab64ab6d7bf admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:21.467833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8f2ad-b34a-4cd1-8da6-fab64ab6d7bf admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:21.472442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15145a32-3034-4dc3-a30f-e94e970cb278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:21.473138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15145a32-3034-4dc3-a30f-e94e970cb278 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:21.474144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15145a32-3034-4dc3-a30f-e94e970cb278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:21.474144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15145a32-3034-4dc3-a30f-e94e970cb278 admin admin] PATH_INFO: `/v3/projects/a5bb9dbcb62a4de58c1c6d7d6324a8e4/users/40d84f8a05f44a71977cc30c6b399d34/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:21.491201 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15145a32-3034-4dc3-a30f-e94e970cb278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5bb9dbcb62a4de58c1c6d7d6324a8e4, user_id=40d84f8a05f44a71977cc30c6b399d34, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:21.494385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15145a32-3034-4dc3-a30f-e94e970cb278 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:21.511710 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 938/1880] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:21 2022] PUT /identity/v3/projects/a5bb9dbcb62a4de58c1c6d7d6324a8e4/users/40d84f8a05f44a71977cc30c6b399d34/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:21.512862 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f32c0-7e01-4197-a71e-cbb7e3262960 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:21.523560 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f32c0-7e01-4197-a71e-cbb7e3262960 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:21.581409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba9be8-850c-4828-a1b9-4de8541c6b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:21.581987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba9be8-850c-4828-a1b9-4de8541c6b45 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:21.582241 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba9be8-850c-4828-a1b9-4de8541c6b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:21.582525 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba9be8-850c-4828-a1b9-4de8541c6b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:21.583169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba9be8-850c-4828-a1b9-4de8541c6b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:21.584808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba9be8-850c-4828-a1b9-4de8541c6b45 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:21.605936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 943/1881] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:32:21.607145 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15145a32-3034-4dc3-a30f-e94e970cb278 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:21.623452 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba9be8-850c-4828-a1b9-4de8541c6b45 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:21.625348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-95ba9be8-850c-4828-a1b9-4de8541c6b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:21.631598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15145a32-3034-4dc3-a30f-e94e970cb278 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:21.636965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260382f-4129-41ed-a69d-38113d67dab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:21.637642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260382f-4129-41ed-a69d-38113d67dab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:21.637921 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260382f-4129-41ed-a69d-38113d67dab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:21.638190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260382f-4129-41ed-a69d-38113d67dab5 admin admin] PATH_INFO: `/v3/projects/a5bb9dbcb62a4de58c1c6d7d6324a8e4/users/40d84f8a05f44a71977cc30c6b399d34/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:21.641081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260382f-4129-41ed-a69d-38113d67dab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5bb9dbcb62a4de58c1c6d7d6324a8e4, user_id=40d84f8a05f44a71977cc30c6b399d34, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:21.644051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260382f-4129-41ed-a69d-38113d67dab5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:21.656177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 939/1882] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:21 2022] PUT /identity/v3/projects/a5bb9dbcb62a4de58c1c6d7d6324a8e4/users/40d84f8a05f44a71977cc30c6b399d34/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:21.656829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260382f-4129-41ed-a69d-38113d67dab5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:21.658630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5260382f-4129-41ed-a69d-38113d67dab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:21.725551 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba9be8-850c-4828-a1b9-4de8541c6b45 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:21.764178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260382f-4129-41ed-a69d-38113d67dab5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:21.769012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791db58-f19a-431b-b055-13ed51f453ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:21.769631 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791db58-f19a-431b-b055-13ed51f453ca service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:21.769905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791db58-f19a-431b-b055-13ed51f453ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:21.770587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791db58-f19a-431b-b055-13ed51f453ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:21.773019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a791db58-f19a-431b-b055-13ed51f453ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:21.788460 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791db58-f19a-431b-b055-13ed51f453ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:21.790527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791db58-f19a-431b-b055-13ed51f453ca service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:21.800854 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 940/1883] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:21.803121 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791db58-f19a-431b-b055-13ed51f453ca service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:21.811057 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791db58-f19a-431b-b055-13ed51f453ca service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:21.815061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8a751-bae2-4695-a44a-cb490330c77b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:21.815694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60015d-e292-4ce5-a022-e6c846eb899c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:21.815851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8a751-bae2-4695-a44a-cb490330c77b service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:21.816108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8a751-bae2-4695-a44a-cb490330c77b service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:21.816404 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60015d-e292-4ce5-a022-e6c846eb899c service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:21.816512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8a751-bae2-4695-a44a-cb490330c77b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:21.816822 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60015d-e292-4ce5-a022-e6c846eb899c service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:21.817349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60015d-e292-4ce5-a022-e6c846eb899c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:21.824688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8a751-bae2-4695-a44a-cb490330c77b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:21.826079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8a751-bae2-4695-a44a-cb490330c77b service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:21.827994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60015d-e292-4ce5-a022-e6c846eb899c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:21.829609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60015d-e292-4ce5-a022-e6c846eb899c service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:21.831656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:21.836920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 944/1884] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:21.837754 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8a751-bae2-4695-a44a-cb490330c77b service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:21.839563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 941/1885] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:21.840977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30b83d-52e6-405b-9c47-050dab90a92d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:21.841328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30b83d-52e6-405b-9c47-050dab90a92d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:21.841672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30b83d-52e6-405b-9c47-050dab90a92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:21.849678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8a751-bae2-4695-a44a-cb490330c77b service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:21.855138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb09e44-5da8-4a9d-8850-22d81b4ec120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:21.857193 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb09e44-5da8-4a9d-8850-22d81b4ec120 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:21.859021 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb09e44-5da8-4a9d-8850-22d81b4ec120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:21.859310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb09e44-5da8-4a9d-8850-22d81b4ec120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:21.874327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ca30b83d-52e6-405b-9c47-050dab90a92d None None] MFA Rules not processed for user `40d84f8a05f44a71977cc30c6b399d34`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:21.877638 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb09e44-5da8-4a9d-8850-22d81b4ec120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:21.878689 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ca30b83d-52e6-405b-9c47-050dab90a92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:21.880851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb09e44-5da8-4a9d-8850-22d81b4ec120 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:21.893431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 945/1886] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:21.964181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 942/1887] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:21.977974 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb09e44-5da8-4a9d-8850-22d81b4ec120 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:21.980245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-3bb09e44-5da8-4a9d-8850-22d81b4ec120 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:22.069354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb09e44-5da8-4a9d-8850-22d81b4ec120 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:22.074406 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c3f3d-022f-41a0-87ec-11369463d321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:22.075501 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c3f3d-022f-41a0-87ec-11369463d321 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:22.076050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c3f3d-022f-41a0-87ec-11369463d321 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:22.076634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c3f3d-022f-41a0-87ec-11369463d321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:22.155813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c3f3d-022f-41a0-87ec-11369463d321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:22.157645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c3f3d-022f-41a0-87ec-11369463d321 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:22.168570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 946/1888] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:23.918333 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b3b46-ced7-461f-b10a-8e8b16b52f3c None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:23.918951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b3b46-ced7-461f-b10a-8e8b16b52f3c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:23.919515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b3b46-ced7-461f-b10a-8e8b16b52f3c None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:23.921008 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 943/1889] 10.222.0.23 () {54 vars in 921 bytes} [Fri Sep 9 16:32:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:32:23.943246 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0441335-1c90-45c6-850d-ac866f3a428e None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:23.943810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0441335-1c90-45c6-850d-ac866f3a428e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:23.944022 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0441335-1c90-45c6-850d-ac866f3a428e None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:23.945824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 947/1890] 10.222.0.23 () {54 vars in 921 bytes} [Fri Sep 9 16:32:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:32:23.949099 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be320a-bed7-4e38-8ee5-39643b9015d5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:23.949512 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be320a-bed7-4e38-8ee5-39643b9015d5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:23.949885 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be320a-bed7-4e38-8ee5-39643b9015d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:23.972251 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-95be320a-bed7-4e38-8ee5-39643b9015d5 None None] MFA Rules not processed for user `152a41ad598b4b01b1881d928c53c848`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:23.973670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-95be320a-bed7-4e38-8ee5-39643b9015d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:23.977324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879b6fa-21f7-4b3f-9425-91c75831c84c None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:23.978375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879b6fa-21f7-4b3f-9425-91c75831c84c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:23.978375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879b6fa-21f7-4b3f-9425-91c75831c84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:23.985445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:24.012298 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-1879b6fa-21f7-4b3f-9425-91c75831c84c None None] MFA Rules not processed for user `152a41ad598b4b01b1881d928c53c848`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:24.013715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1879b6fa-21f7-4b3f-9425-91c75831c84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:24.096879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 944/1891] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Sep 9 16:32:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:24.109924 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be320a-bed7-4e38-8ee5-39643b9015d5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:24.112667 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-95be320a-bed7-4e38-8ee5-39643b9015d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:24.133358 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be320a-bed7-4e38-8ee5-39643b9015d5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:24.140145 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ace83-470e-4809-8dbb-4e81a24711a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:24.141247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ace83-470e-4809-8dbb-4e81a24711a0 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:24.141716 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ace83-470e-4809-8dbb-4e81a24711a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:24.142158 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ace83-470e-4809-8dbb-4e81a24711a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:24.143425 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 948/1892] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Sep 9 16:32:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:24.160280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879b6fa-21f7-4b3f-9425-91c75831c84c None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:24.161714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1879b6fa-21f7-4b3f-9425-91c75831c84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:24.165859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ace83-470e-4809-8dbb-4e81a24711a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:24.167544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ace83-470e-4809-8dbb-4e81a24711a0 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:24.180529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 945/1893] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:24.181193 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879b6fa-21f7-4b3f-9425-91c75831c84c None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:24.187543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95a9c6-0164-4345-9245-8342724d9588 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:24.188408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95a9c6-0164-4345-9245-8342724d9588 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:24.189556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95a9c6-0164-4345-9245-8342724d9588 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:24.189878 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95a9c6-0164-4345-9245-8342724d9588 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:24.203273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95a9c6-0164-4345-9245-8342724d9588 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:24.205751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95a9c6-0164-4345-9245-8342724d9588 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:24.220900 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 949/1894] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:29.255396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ace83-470e-4809-8dbb-4e81a24711a0 service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:29.265236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ace83-470e-4809-8dbb-4e81a24711a0 service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:29.269241 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87ccb0-26c6-45c6-ba29-8d5a9e9f11f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:29.269844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87ccb0-26c6-45c6-ba29-8d5a9e9f11f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:29.270116 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87ccb0-26c6-45c6-ba29-8d5a9e9f11f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:29.270382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87ccb0-26c6-45c6-ba29-8d5a9e9f11f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:29.329186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87ccb0-26c6-45c6-ba29-8d5a9e9f11f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:29.330713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87ccb0-26c6-45c6-ba29-8d5a9e9f11f0 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:29.339744 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 946/1895] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:31.620176 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95a9c6-0164-4345-9245-8342724d9588 service nova] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:31.644970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95a9c6-0164-4345-9245-8342724d9588 service nova] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:31.649795 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a3688-f7b5-40ac-af62-a1901b515881 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:31.651733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a3688-f7b5-40ac-af62-a1901b515881 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:31.652118 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a3688-f7b5-40ac-af62-a1901b515881 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:31.652578 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a3688-f7b5-40ac-af62-a1901b515881 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:31.664403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a3688-f7b5-40ac-af62-a1901b515881 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:31.666355 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a3688-f7b5-40ac-af62-a1901b515881 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:31.678998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 950/1896] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:33.711520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87ccb0-26c6-45c6-ba29-8d5a9e9f11f0 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:33.713844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2a87ccb0-26c6-45c6-ba29-8d5a9e9f11f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:33.807556 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87ccb0-26c6-45c6-ba29-8d5a9e9f11f0 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:33.812445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a881c-2c45-4f41-b660-3d656764a0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:33.813082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a881c-2c45-4f41-b660-3d656764a0e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:33.813388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a881c-2c45-4f41-b660-3d656764a0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:33.813629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a881c-2c45-4f41-b660-3d656764a0e0 admin admin] PATH_INFO: `/v3/users/39e0161d99f04f3697e6f3925310ea26` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:33.814793 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a881c-2c45-4f41-b660-3d656764a0e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39e0161d99f04f3697e6f3925310ea26)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:33.816737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a881c-2c45-4f41-b660-3d656764a0e0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:33.845889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:33.867391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f3a881c-2c45-4f41-b660-3d656764a0e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39e0161d99f04f3697e6f3925310ea26'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:33.868161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f3a881c-2c45-4f41-b660-3d656764a0e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39e0161d99f04f3697e6f3925310ea26'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:33.868889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f3a881c-2c45-4f41-b660-3d656764a0e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39e0161d99f04f3697e6f3925310ea26'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:33.876647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f3a881c-2c45-4f41-b660-3d656764a0e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39e0161d99f04f3697e6f3925310ea26'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:33.889364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f3a881c-2c45-4f41-b660-3d656764a0e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39e0161d99f04f3697e6f3925310ea26'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:33.908553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 947/1897] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:33 2022] DELETE /identity/v3/users/39e0161d99f04f3697e6f3925310ea26 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:33.913243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c1e58-3603-44bf-a2d0-a6ac30104409 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:33.913714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c1e58-3603-44bf-a2d0-a6ac30104409 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:33.913810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c1e58-3603-44bf-a2d0-a6ac30104409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:33.931281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:33.938001 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d96c1e58-3603-44bf-a2d0-a6ac30104409 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:33.939274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d96c1e58-3603-44bf-a2d0-a6ac30104409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:34.009141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 951/1898] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:34.019282 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a881c-2c45-4f41-b660-3d656764a0e0 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:34.020879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4f3a881c-2c45-4f41-b660-3d656764a0e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:34.078789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a881c-2c45-4f41-b660-3d656764a0e0 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:34.082589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18eed-2eb7-4053-b772-d421e9fbe81a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:34.083308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18eed-2eb7-4053-b772-d421e9fbe81a service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:34.083589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18eed-2eb7-4053-b772-d421e9fbe81a service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:34.083861 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18eed-2eb7-4053-b772-d421e9fbe81a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:34.093273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c18eed-2eb7-4053-b772-d421e9fbe81a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:34.094771 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c18eed-2eb7-4053-b772-d421e9fbe81a service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:34.104172 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 948/1899] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:34.475399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c1e58-3603-44bf-a2d0-a6ac30104409 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:34.477080 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d96c1e58-3603-44bf-a2d0-a6ac30104409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:34.497454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c1e58-3603-44bf-a2d0-a6ac30104409 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:34.525577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:34.526253 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:34.526556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:34.526846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] PATH_INFO: `/v3/projects/670e32b4b2a44d0c83d60258d73bdc9e` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:34.530056 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=670e32b4b2a44d0c83d60258d73bdc9e)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:34.538550 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:34.663251 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670e32b4b2a44d0c83d60258d73bdc9e', 'request_id': 'req-cacbff37-efc5-44dd-8afc-dd6681a1a045'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:34.675746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670e32b4b2a44d0c83d60258d73bdc9e', 'request_id': 'req-cacbff37-efc5-44dd-8afc-dd6681a1a045'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:34.700835 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670e32b4b2a44d0c83d60258d73bdc9e', 'request_id': 'req-cacbff37-efc5-44dd-8afc-dd6681a1a045'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:34.732887 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] The token cache is being invalidate because project 670e32b4b2a44d0c83d60258d73bdc9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:34.733242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:34.734093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:34.737896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 952/1900] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:34 2022] DELETE /identity/v3/projects/670e32b4b2a44d0c83d60258d73bdc9e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:34.741789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18eed-2eb7-4053-b772-d421e9fbe81a service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:34.756039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18eed-2eb7-4053-b772-d421e9fbe81a service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:34.901835 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8267b80-4c89-4143-a47f-7c7fadce3639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:34.902581 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8267b80-4c89-4143-a47f-7c7fadce3639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:34.902840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8267b80-4c89-4143-a47f-7c7fadce3639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:34.903144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8267b80-4c89-4143-a47f-7c7fadce3639 admin admin] PATH_INFO: `/v3/users/e6df75c1ec41456c809638011ea4ba19` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:34.910912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8267b80-4c89-4143-a47f-7c7fadce3639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6df75c1ec41456c809638011ea4ba19)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:34.913153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8267b80-4c89-4143-a47f-7c7fadce3639 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:35.052609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c8267b80-4c89-4143-a47f-7c7fadce3639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6df75c1ec41456c809638011ea4ba19'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.053310 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c8267b80-4c89-4143-a47f-7c7fadce3639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6df75c1ec41456c809638011ea4ba19'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.053921 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c8267b80-4c89-4143-a47f-7c7fadce3639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6df75c1ec41456c809638011ea4ba19'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.056587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:35.066620 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c8267b80-4c89-4143-a47f-7c7fadce3639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6df75c1ec41456c809638011ea4ba19'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.101837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c8267b80-4c89-4143-a47f-7c7fadce3639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6df75c1ec41456c809638011ea4ba19'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.135361 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 949/1901] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:34 2022] DELETE /identity/v3/users/e6df75c1ec41456c809638011ea4ba19 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:35.252642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:35.257420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:35.293552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacbff37-efc5-44dd-8afc-dd6681a1a045 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:35.392916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94e37-9d3e-463b-ad92-54a86b07615c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:35.393824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94e37-9d3e-463b-ad92-54a86b07615c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:35.394229 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94e37-9d3e-463b-ad92-54a86b07615c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:35.395197 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94e37-9d3e-463b-ad92-54a86b07615c admin admin] PATH_INFO: `/v3/users/252f2312bbfb4dcfba616f7051f6a124` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:35.396699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd94e37-9d3e-463b-ad92-54a86b07615c admin admin] RBAC: Authorizing `identity:delete_user(user_id=252f2312bbfb4dcfba616f7051f6a124)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:35.399043 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd94e37-9d3e-463b-ad92-54a86b07615c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:35.453047 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:35.474604 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-8dd94e37-9d3e-463b-ad92-54a86b07615c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '252f2312bbfb4dcfba616f7051f6a124'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.475728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-8dd94e37-9d3e-463b-ad92-54a86b07615c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '252f2312bbfb4dcfba616f7051f6a124'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.476421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-8dd94e37-9d3e-463b-ad92-54a86b07615c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '252f2312bbfb4dcfba616f7051f6a124'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.483527 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-8dd94e37-9d3e-463b-ad92-54a86b07615c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '252f2312bbfb4dcfba616f7051f6a124'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.497789 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-8dd94e37-9d3e-463b-ad92-54a86b07615c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '252f2312bbfb4dcfba616f7051f6a124'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.515127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 953/1902] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:35 2022] DELETE /identity/v3/users/252f2312bbfb4dcfba616f7051f6a124 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:35.519977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8afa0-bb86-4cb4-8cb2-610e0a86773f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:35.520271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8afa0-bb86-4cb4-8cb2-610e0a86773f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:35.520645 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8afa0-bb86-4cb4-8cb2-610e0a86773f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:35.540849 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-67f8afa0-bb86-4cb4-8cb2-610e0a86773f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:35.542083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-67f8afa0-bb86-4cb4-8cb2-610e0a86773f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:35.611612 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 950/1903] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:35.621673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94e37-9d3e-463b-ad92-54a86b07615c admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:35.623308 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8dd94e37-9d3e-463b-ad92-54a86b07615c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:35.694764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94e37-9d3e-463b-ad92-54a86b07615c admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:35.700467 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8afa0-bb86-4cb4-8cb2-610e0a86773f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:35.703709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-67f8afa0-bb86-4cb4-8cb2-610e0a86773f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:35.704783 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc057f8-6931-44d8-bef9-c85448f98625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:35.705619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc057f8-6931-44d8-bef9-c85448f98625 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:35.705988 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc057f8-6931-44d8-bef9-c85448f98625 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:35.706350 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc057f8-6931-44d8-bef9-c85448f98625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:35.719317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc057f8-6931-44d8-bef9-c85448f98625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:35.720908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc057f8-6931-44d8-bef9-c85448f98625 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:35.729629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8afa0-bb86-4cb4-8cb2-610e0a86773f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:35.732231 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 954/1904] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:35.738289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:35.739078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:35.739481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:35.739817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6 admin admin] PATH_INFO: `/v3/projects/662a6e50bec5432f92461adf17d6252b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:35.741079 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=662a6e50bec5432f92461adf17d6252b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:35.743202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:35.806934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '662a6e50bec5432f92461adf17d6252b', 'request_id': 'req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.816538 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '662a6e50bec5432f92461adf17d6252b', 'request_id': 'req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.824407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '662a6e50bec5432f92461adf17d6252b', 'request_id': 'req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.846132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6 admin admin] The token cache is being invalidate because project 662a6e50bec5432f92461adf17d6252b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:35.846442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.847786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f7ea8a7-b2b8-4d2f-a145-15039cf156d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:35.851748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 951/1905] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:35 2022] DELETE /identity/v3/projects/662a6e50bec5432f92461adf17d6252b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:35.856707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb038fe8-d48b-4c30-8275-270737fde2d1 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:35.857345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb038fe8-d48b-4c30-8275-270737fde2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:35.857717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb038fe8-d48b-4c30-8275-270737fde2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:35.879387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-eb038fe8-d48b-4c30-8275-270737fde2d1 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:35.880771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-eb038fe8-d48b-4c30-8275-270737fde2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:35.959939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 955/1906] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:35.967341 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e886deb-3f12-4d13-a934-4f6f1045a8dc None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:35.967622 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e886deb-3f12-4d13-a934-4f6f1045a8dc None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:35.967896 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e886deb-3f12-4d13-a934-4f6f1045a8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:35.989399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0e886deb-3f12-4d13-a934-4f6f1045a8dc None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:35.990806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0e886deb-3f12-4d13-a934-4f6f1045a8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:36.000471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 952/1907] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:36.004814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb038fe8-d48b-4c30-8275-270737fde2d1 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:36.018232 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb038fe8-d48b-4c30-8275-270737fde2d1 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:36.041159 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e2032-b04b-454b-9a57-95790f147bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:36.042198 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e2032-b04b-454b-9a57-95790f147bec admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.042402 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e2032-b04b-454b-9a57-95790f147bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.042907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e2032-b04b-454b-9a57-95790f147bec admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.043675 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e2032-b04b-454b-9a57-95790f147bec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:36.043978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e2032-b04b-454b-9a57-95790f147bec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:36.045601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e2032-b04b-454b-9a57-95790f147bec admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:36.059295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 956/1908] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:32:36.066719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e886deb-3f12-4d13-a934-4f6f1045a8dc None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:36.078929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e886deb-3f12-4d13-a934-4f6f1045a8dc None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:36.084394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7850f4-3e15-4d9f-8629-beb160f2b4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:36.085090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7850f4-3e15-4d9f-8629-beb160f2b4cd admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.085438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7850f4-3e15-4d9f-8629-beb160f2b4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.085778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7850f4-3e15-4d9f-8629-beb160f2b4cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.086636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7850f4-3e15-4d9f-8629-beb160f2b4cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:36.089028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7850f4-3e15-4d9f-8629-beb160f2b4cd admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:36.133824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 953/1909] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:36 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:36.135400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e2032-b04b-454b-9a57-95790f147bec admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:36.146460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e2032-b04b-454b-9a57-95790f147bec admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:36.188444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:36.208268 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6a000-f302-41d7-b469-f818065ded16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:36.208884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6a000-f302-41d7-b469-f818065ded16 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.209162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6a000-f302-41d7-b469-f818065ded16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.209430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6a000-f302-41d7-b469-f818065ded16 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.210065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6a000-f302-41d7-b469-f818065ded16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:36.212082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6a000-f302-41d7-b469-f818065ded16 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:36.225452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7850f4-3e15-4d9f-8629-beb160f2b4cd admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:36.227308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4d7850f4-3e15-4d9f-8629-beb160f2b4cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:36.240117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 957/1910] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:36 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:36.243895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6a000-f302-41d7-b469-f818065ded16 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:36.252668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7850f4-3e15-4d9f-8629-beb160f2b4cd admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:36.253939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6a000-f302-41d7-b469-f818065ded16 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:36.259665 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f6b7d-375d-4492-ba77-d5031a61cdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:36.259798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:36.260393 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.260497 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f6b7d-375d-4492-ba77-d5031a61cdb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.260670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.260771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f6b7d-375d-4492-ba77-d5031a61cdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.260935 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] PATH_INFO: `/v3/projects/f187443561a4466786dc67c3b5b1b6da` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.261684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f6b7d-375d-4492-ba77-d5031a61cdb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.262053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f187443561a4466786dc67c3b5b1b6da)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:36.262358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f6b7d-375d-4492-ba77-d5031a61cdb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:36.263979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f6b7d-375d-4492-ba77-d5031a61cdb9 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:36.264078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:36.275634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 958/1911] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:32:36.279259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f6b7d-375d-4492-ba77-d5031a61cdb9 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:36.281941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:36.289837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f6b7d-375d-4492-ba77-d5031a61cdb9 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:36.295562 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ca86e-bf5c-4342-a345-827a7eaf52a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:36.296334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ca86e-bf5c-4342-a345-827a7eaf52a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.296804 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ca86e-bf5c-4342-a345-827a7eaf52a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.297174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ca86e-bf5c-4342-a345-827a7eaf52a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.297851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ca86e-bf5c-4342-a345-827a7eaf52a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:36.299976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ca86e-bf5c-4342-a345-827a7eaf52a9 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:36.312515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 959/1912] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:32:36.316531 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ca86e-bf5c-4342-a345-827a7eaf52a9 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:36.325018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f187443561a4466786dc67c3b5b1b6da', 'request_id': 'req-f181cadb-f706-44f3-b479-5a7552554922'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.327815 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ca86e-bf5c-4342-a345-827a7eaf52a9 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:36.331872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f187443561a4466786dc67c3b5b1b6da', 'request_id': 'req-f181cadb-f706-44f3-b479-5a7552554922'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.345078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f187443561a4466786dc67c3b5b1b6da', 'request_id': 'req-f181cadb-f706-44f3-b479-5a7552554922'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.363155 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] The token cache is being invalidate because project f187443561a4466786dc67c3b5b1b6da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:36.363700 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.364782 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.369777 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 954/1913] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:36 2022] DELETE /identity/v3/projects/f187443561a4466786dc67c3b5b1b6da => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:36.371646 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:36.373279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:36.392969 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e50830-ad12-447a-9d3f-9af7e568e8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:36.393715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e50830-ad12-447a-9d3f-9af7e568e8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.394111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e50830-ad12-447a-9d3f-9af7e568e8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.394516 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e50830-ad12-447a-9d3f-9af7e568e8a5 admin admin] PATH_INFO: `/v3/projects/0a0e68e24a3649f0bc00dab3192fdf11/users/b8457e8344bf4b3983aa2fbceeb57bc1/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.416927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e50830-ad12-447a-9d3f-9af7e568e8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0e68e24a3649f0bc00dab3192fdf11, user_id=b8457e8344bf4b3983aa2fbceeb57bc1, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:36.423220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e50830-ad12-447a-9d3f-9af7e568e8a5 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:36.452079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 960/1914] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:36 2022] PUT /identity/v3/projects/0a0e68e24a3649f0bc00dab3192fdf11/users/b8457e8344bf4b3983aa2fbceeb57bc1/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:36.452760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181cadb-f706-44f3-b479-5a7552554922 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:36.469768 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af4525-9c0f-4e26-ae1c-d0af53dcaace None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.470053 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af4525-9c0f-4e26-ae1c-d0af53dcaace None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.470328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af4525-9c0f-4e26-ae1c-d0af53dcaace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.515387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-05af4525-9c0f-4e26-ae1c-d0af53dcaace None None] MFA Rules not processed for user `b8457e8344bf4b3983aa2fbceeb57bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:36.516955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-05af4525-9c0f-4e26-ae1c-d0af53dcaace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:36.546290 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a0cc2-9c6f-4ab2-a34e-0cfe2b056558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:36.546987 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a0cc2-9c6f-4ab2-a34e-0cfe2b056558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.547266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a0cc2-9c6f-4ab2-a34e-0cfe2b056558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.547540 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a0cc2-9c6f-4ab2-a34e-0cfe2b056558 admin admin] PATH_INFO: `/v3/users/2cf2cef9de4244e59477f48993a2e2d6` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.549051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a0cc2-9c6f-4ab2-a34e-0cfe2b056558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cf2cef9de4244e59477f48993a2e2d6)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:36.551089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a0cc2-9c6f-4ab2-a34e-0cfe2b056558 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:36.593832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 961/1915] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:36.594607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af4525-9c0f-4e26-ae1c-d0af53dcaace None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:36.596094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-05af4525-9c0f-4e26-ae1c-d0af53dcaace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:36.603930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-373a0cc2-9c6f-4ab2-a34e-0cfe2b056558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cf2cef9de4244e59477f48993a2e2d6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.604846 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-373a0cc2-9c6f-4ab2-a34e-0cfe2b056558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cf2cef9de4244e59477f48993a2e2d6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.605733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-373a0cc2-9c6f-4ab2-a34e-0cfe2b056558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cf2cef9de4244e59477f48993a2e2d6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.611109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af4525-9c0f-4e26-ae1c-d0af53dcaace None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:36.611643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:36.613926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-373a0cc2-9c6f-4ab2-a34e-0cfe2b056558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cf2cef9de4244e59477f48993a2e2d6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.617119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7420ee5-90a9-4ead-b9d2-659cf23eb4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:36.617894 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7420ee5-90a9-4ead-b9d2-659cf23eb4e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.618177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7420ee5-90a9-4ead-b9d2-659cf23eb4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.618593 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7420ee5-90a9-4ead-b9d2-659cf23eb4e1 admin admin] PATH_INFO: `/v3/users/47976541b90846fc9c8b9ce5743bcc4c` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.619908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7420ee5-90a9-4ead-b9d2-659cf23eb4e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47976541b90846fc9c8b9ce5743bcc4c)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:36.621848 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7420ee5-90a9-4ead-b9d2-659cf23eb4e1 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:36.631632 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-373a0cc2-9c6f-4ab2-a34e-0cfe2b056558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cf2cef9de4244e59477f48993a2e2d6'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.639880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:36.649165 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 955/1916] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:36 2022] DELETE /identity/v3/users/2cf2cef9de4244e59477f48993a2e2d6 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:36.649989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a0cc2-9c6f-4ab2-a34e-0cfe2b056558 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:36.651702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-373a0cc2-9c6f-4ab2-a34e-0cfe2b056558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:36.694286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f7420ee5-90a9-4ead-b9d2-659cf23eb4e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47976541b90846fc9c8b9ce5743bcc4c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.695767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f7420ee5-90a9-4ead-b9d2-659cf23eb4e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47976541b90846fc9c8b9ce5743bcc4c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.696780 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f7420ee5-90a9-4ead-b9d2-659cf23eb4e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47976541b90846fc9c8b9ce5743bcc4c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.704084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f7420ee5-90a9-4ead-b9d2-659cf23eb4e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47976541b90846fc9c8b9ce5743bcc4c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.719068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f7420ee5-90a9-4ead-b9d2-659cf23eb4e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47976541b90846fc9c8b9ce5743bcc4c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:36.733481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a0cc2-9c6f-4ab2-a34e-0cfe2b056558 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:36.738358 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480fdd2-6d71-47cb-b5ed-109a8ca0b0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:36.739008 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480fdd2-6d71-47cb-b5ed-109a8ca0b0fb admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.739316 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480fdd2-6d71-47cb-b5ed-109a8ca0b0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.739588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480fdd2-6d71-47cb-b5ed-109a8ca0b0fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.742641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 962/1917] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:36 2022] DELETE /identity/v3/users/47976541b90846fc9c8b9ce5743bcc4c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:36.742766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480fdd2-6d71-47cb-b5ed-109a8ca0b0fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:36.742766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480fdd2-6d71-47cb-b5ed-109a8ca0b0fb admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:36.744957 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c9e8d-de0c-4045-a0c6-59c59cd538ea None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.745276 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c9e8d-de0c-4045-a0c6-59c59cd538ea None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.745607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c9e8d-de0c-4045-a0c6-59c59cd538ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.765261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-7c0c9e8d-de0c-4045-a0c6-59c59cd538ea None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:36.766694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7c0c9e8d-de0c-4045-a0c6-59c59cd538ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:36.772642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 956/1918] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:36 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:36.776822 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480fdd2-6d71-47cb-b5ed-109a8ca0b0fb admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:36.778424 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3480fdd2-6d71-47cb-b5ed-109a8ca0b0fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:36.838742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 963/1919] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:36.849664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c9e8d-de0c-4045-a0c6-59c59cd538ea None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:36.851285 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7c0c9e8d-de0c-4045-a0c6-59c59cd538ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:36.855217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480fdd2-6d71-47cb-b5ed-109a8ca0b0fb admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:36.859895 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de2338-128d-42ab-b68f-3528b74d2a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:36.860607 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de2338-128d-42ab-b68f-3528b74d2a0d admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.861007 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de2338-128d-42ab-b68f-3528b74d2a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.861602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de2338-128d-42ab-b68f-3528b74d2a0d admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.862789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de2338-128d-42ab-b68f-3528b74d2a0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:36.864935 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de2338-128d-42ab-b68f-3528b74d2a0d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:36.891808 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 957/1920] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:36 2022] POST /identity/v3/users => generated 300 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:32:36.896753 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de2338-128d-42ab-b68f-3528b74d2a0d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:36.908694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de2338-128d-42ab-b68f-3528b74d2a0d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:36.915576 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c9e8d-de0c-4045-a0c6-59c59cd538ea None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:36.924216 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2672c5-506f-4371-9a4d-00c7a6d3ceb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:36.925260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2672c5-506f-4371-9a4d-00c7a6d3ceb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.925410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2672c5-506f-4371-9a4d-00c7a6d3ceb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.925639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2672c5-506f-4371-9a4d-00c7a6d3ceb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.926483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2672c5-506f-4371-9a4d-00c7a6d3ceb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:36.928492 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2672c5-506f-4371-9a4d-00c7a6d3ceb3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:36.944494 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 958/1921] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:32:36.949265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2672c5-506f-4371-9a4d-00c7a6d3ceb3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:36.960541 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2672c5-506f-4371-9a4d-00c7a6d3ceb3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:36.967083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9917ba6-5e24-432f-9d7b-0ddd4a9dba15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:36.967867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9917ba6-5e24-432f-9d7b-0ddd4a9dba15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.968208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9917ba6-5e24-432f-9d7b-0ddd4a9dba15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.968913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9917ba6-5e24-432f-9d7b-0ddd4a9dba15 admin admin] PATH_INFO: `/v3/projects/a2d4eb7f5b214dffa52377e04d271569/users/79e00f2a48334805be57af4b6c32e6fb/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:36.980825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:36.985158 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9917ba6-5e24-432f-9d7b-0ddd4a9dba15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d4eb7f5b214dffa52377e04d271569, user_id=79e00f2a48334805be57af4b6c32e6fb, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:36.987472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204a499-20de-45c5-955c-1b6bd32dcecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:36.988277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204a499-20de-45c5-955c-1b6bd32dcecb service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:36.988454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9917ba6-5e24-432f-9d7b-0ddd4a9dba15 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:36.989329 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204a499-20de-45c5-955c-1b6bd32dcecb service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:36.989649 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204a499-20de-45c5-955c-1b6bd32dcecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:37.000561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204a499-20de-45c5-955c-1b6bd32dcecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:37.002041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204a499-20de-45c5-955c-1b6bd32dcecb service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:37.003013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 959/1922] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:36 2022] PUT /identity/v3/projects/a2d4eb7f5b214dffa52377e04d271569/users/79e00f2a48334805be57af4b6c32e6fb/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:37.003925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9917ba6-5e24-432f-9d7b-0ddd4a9dba15 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:37.005617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a9917ba6-5e24-432f-9d7b-0ddd4a9dba15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:37.012616 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 964/1923] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:37.013441 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204a499-20de-45c5-955c-1b6bd32dcecb service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:37.023144 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204a499-20de-45c5-955c-1b6bd32dcecb service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:37.024830 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:37.108124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1af99-78fd-454f-8970-c7742ca5f30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:37.109016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1af99-78fd-454f-8970-c7742ca5f30b admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:37.109350 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1af99-78fd-454f-8970-c7742ca5f30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:37.110004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1af99-78fd-454f-8970-c7742ca5f30b admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:37.110881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1af99-78fd-454f-8970-c7742ca5f30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:37.113030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1af99-78fd-454f-8970-c7742ca5f30b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:37.120494 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9917ba6-5e24-432f-9d7b-0ddd4a9dba15 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:37.126301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:37.130135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:37.130271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 965/1924] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:37.130366 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:37.130366 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8 admin admin] PATH_INFO: `/v3/projects/9eb6c8a49195428abd22eca13da534e0` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:37.130366 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb6c8a49195428abd22eca13da534e0)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:37.130930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:37.131968 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1af99-78fd-454f-8970-c7742ca5f30b admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:37.145686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1af99-78fd-454f-8970-c7742ca5f30b admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:37.152943 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb0e18-fd07-46ac-b787-96ea4d94d18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:37.153771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0e18-fd07-46ac-b787-96ea4d94d18b admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:37.154122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0e18-fd07-46ac-b787-96ea4d94d18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:37.154555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0e18-fd07-46ac-b787-96ea4d94d18b admin admin] PATH_INFO: `/v3/projects/a2d4eb7f5b214dffa52377e04d271569/users/79e00f2a48334805be57af4b6c32e6fb/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:37.156948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb0e18-fd07-46ac-b787-96ea4d94d18b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d4eb7f5b214dffa52377e04d271569, user_id=79e00f2a48334805be57af4b6c32e6fb, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:37.160038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb0e18-fd07-46ac-b787-96ea4d94d18b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:37.183806 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 966/1925] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:37 2022] PUT /identity/v3/projects/a2d4eb7f5b214dffa52377e04d271569/users/79e00f2a48334805be57af4b6c32e6fb/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:37.185333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b38b9-6a27-4ff6-a434-5803f6bc1b06 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:37.185622 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b38b9-6a27-4ff6-a434-5803f6bc1b06 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:37.185895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b38b9-6a27-4ff6-a434-5803f6bc1b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:37.226408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb6c8a49195428abd22eca13da534e0', 'request_id': 'req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.233740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-ac9b38b9-6a27-4ff6-a434-5803f6bc1b06 None None] MFA Rules not processed for user `79e00f2a48334805be57af4b6c32e6fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:37.234751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ac9b38b9-6a27-4ff6-a434-5803f6bc1b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:37.239234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb6c8a49195428abd22eca13da534e0', 'request_id': 'req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.246859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb6c8a49195428abd22eca13da534e0', 'request_id': 'req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.270106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8 admin admin] The token cache is being invalidate because project 9eb6c8a49195428abd22eca13da534e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:37.270574 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.272773 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e9b7d5e3-697c-4c55-b0b1-ce5da3d28ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.275786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 960/1926] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:37 2022] DELETE /identity/v3/projects/9eb6c8a49195428abd22eca13da534e0 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:37.280569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c027-5731-47f8-86b0-2a7bf0f60f94 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:37.280778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c027-5731-47f8-86b0-2a7bf0f60f94 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:37.281028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c027-5731-47f8-86b0-2a7bf0f60f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:37.319681 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-9a42c027-5731-47f8-86b0-2a7bf0f60f94 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:37.321030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9a42c027-5731-47f8-86b0-2a7bf0f60f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:37.373699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 967/1927] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:37.389635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b38b9-6a27-4ff6-a434-5803f6bc1b06 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:37.391609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ac9b38b9-6a27-4ff6-a434-5803f6bc1b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:37.400371 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 961/1928] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:37.403479 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42c027-5731-47f8-86b0-2a7bf0f60f94 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:37.407637 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9a42c027-5731-47f8-86b0-2a7bf0f60f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:37.418450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b38b9-6a27-4ff6-a434-5803f6bc1b06 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:37.425225 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:37.426763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:37.427052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:37.427324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] PATH_INFO: `/v3/projects/dc185f2afd1246669850674a10e608d0` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:37.429166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc185f2afd1246669850674a10e608d0)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:37.432052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:37.459272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:37.472465 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42c027-5731-47f8-86b0-2a7bf0f60f94 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:37.477060 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b294d5d-2b38-4d9d-b882-f3e0471b1d57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:37.477814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294d5d-2b38-4d9d-b882-f3e0471b1d57 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:37.478255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294d5d-2b38-4d9d-b882-f3e0471b1d57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:37.478611 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294d5d-2b38-4d9d-b882-f3e0471b1d57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:37.489657 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b294d5d-2b38-4d9d-b882-f3e0471b1d57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:37.491325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b294d5d-2b38-4d9d-b882-f3e0471b1d57 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:37.497028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc185f2afd1246669850674a10e608d0', 'request_id': 'req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.506445 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc185f2afd1246669850674a10e608d0', 'request_id': 'req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.508791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:37.518762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 962/1929] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:37.521331 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc185f2afd1246669850674a10e608d0', 'request_id': 'req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.523936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fed5a-4e02-4367-bb0d-f22b43ccbac5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:37.524072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fed5a-4e02-4367-bb0d-f22b43ccbac5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:37.524257 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fed5a-4e02-4367-bb0d-f22b43ccbac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:37.538727 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] The token cache is being invalidate because project dc185f2afd1246669850674a10e608d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:37.539013 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.540777 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.546678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 968/1930] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:37 2022] DELETE /identity/v3/projects/dc185f2afd1246669850674a10e608d0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:37.551557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:37.555956 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:37.566174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-607fed5a-4e02-4367-bb0d-f22b43ccbac5 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:37.567804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-607fed5a-4e02-4367-bb0d-f22b43ccbac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:37.656940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 963/1931] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:37.661298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fed5a-4e02-4367-bb0d-f22b43ccbac5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:37.663057 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-607fed5a-4e02-4367-bb0d-f22b43ccbac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:37.667018 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e465ce7-a5aa-4dbd-b335-c3bf10704ce5 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:37.679607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8c957-9124-41b2-8c96-6691c9fc1cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:37.680244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8c957-9124-41b2-8c96-6691c9fc1cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:37.680524 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8c957-9124-41b2-8c96-6691c9fc1cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:37.681631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8c957-9124-41b2-8c96-6691c9fc1cd7 admin admin] PATH_INFO: `/v3/users/256fb32309a04a2e9261031ac2a4ac19` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:37.686334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8c957-9124-41b2-8c96-6691c9fc1cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=256fb32309a04a2e9261031ac2a4ac19)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:37.688449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8c957-9124-41b2-8c96-6691c9fc1cd7 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:37.696554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fed5a-4e02-4367-bb0d-f22b43ccbac5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:37.711547 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6aba2-15ec-434d-9828-7b133360c8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:37.712817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6aba2-15ec-434d-9828-7b133360c8b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:37.713101 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6aba2-15ec-434d-9828-7b133360c8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:37.713371 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6aba2-15ec-434d-9828-7b133360c8b2 admin admin] PATH_INFO: `/v3/users/b8457e8344bf4b3983aa2fbceeb57bc1` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:37.715653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6aba2-15ec-434d-9828-7b133360c8b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8457e8344bf4b3983aa2fbceeb57bc1)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:37.719498 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6aba2-15ec-434d-9828-7b133360c8b2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:37.761509 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-36d8c957-9124-41b2-8c96-6691c9fc1cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '256fb32309a04a2e9261031ac2a4ac19'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.763323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-36d8c957-9124-41b2-8c96-6691c9fc1cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '256fb32309a04a2e9261031ac2a4ac19'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.765843 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-36d8c957-9124-41b2-8c96-6691c9fc1cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '256fb32309a04a2e9261031ac2a4ac19'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.779182 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-36d8c957-9124-41b2-8c96-6691c9fc1cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '256fb32309a04a2e9261031ac2a4ac19'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.796239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-36d8c957-9124-41b2-8c96-6691c9fc1cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '256fb32309a04a2e9261031ac2a4ac19'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.806993 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d7a6aba2-15ec-434d-9828-7b133360c8b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8457e8344bf4b3983aa2fbceeb57bc1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.808293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d7a6aba2-15ec-434d-9828-7b133360c8b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8457e8344bf4b3983aa2fbceeb57bc1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.809170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d7a6aba2-15ec-434d-9828-7b133360c8b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8457e8344bf4b3983aa2fbceeb57bc1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.811110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 969/1932] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:37 2022] DELETE /identity/v3/users/256fb32309a04a2e9261031ac2a4ac19 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:37.812027 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8c957-9124-41b2-8c96-6691c9fc1cd7 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:37.813426 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-36d8c957-9124-41b2-8c96-6691c9fc1cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:37.823177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d7a6aba2-15ec-434d-9828-7b133360c8b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8457e8344bf4b3983aa2fbceeb57bc1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.839621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d7a6aba2-15ec-434d-9828-7b133360c8b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8457e8344bf4b3983aa2fbceeb57bc1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:37.859652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 964/1933] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:37 2022] DELETE /identity/v3/users/b8457e8344bf4b3983aa2fbceeb57bc1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:37.863649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f57f98-ffd4-4e90-bd74-eaf59eff466a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:37.863935 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f57f98-ffd4-4e90-bd74-eaf59eff466a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:37.864245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f57f98-ffd4-4e90-bd74-eaf59eff466a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:37.898251 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-24f57f98-ffd4-4e90-bd74-eaf59eff466a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:37.899483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-24f57f98-ffd4-4e90-bd74-eaf59eff466a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:37.908289 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8c957-9124-41b2-8c96-6691c9fc1cd7 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:37.913897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:37.960231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:37.972965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 965/1934] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:37.977149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b072897-5bbc-469c-8c6b-72db75ee10d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:37.977775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b072897-5bbc-469c-8c6b-72db75ee10d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:37.978074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b072897-5bbc-469c-8c6b-72db75ee10d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:37.978345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b072897-5bbc-469c-8c6b-72db75ee10d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:37.979020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b072897-5bbc-469c-8c6b-72db75ee10d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:37.979316 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b072897-5bbc-469c-8c6b-72db75ee10d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:37.980917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b072897-5bbc-469c-8c6b-72db75ee10d9 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:37.983390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f57f98-ffd4-4e90-bd74-eaf59eff466a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:37.984749 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-24f57f98-ffd4-4e90-bd74-eaf59eff466a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:38.004932 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 970/1935] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:32:38.009825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b072897-5bbc-469c-8c6b-72db75ee10d9 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:38.012167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-3b072897-5bbc-469c-8c6b-72db75ee10d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:38.027654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b072897-5bbc-469c-8c6b-72db75ee10d9 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:38.037134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665660e-e22b-41ce-92d0-c60f08a65081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:38.037934 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665660e-e22b-41ce-92d0-c60f08a65081 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:38.038218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665660e-e22b-41ce-92d0-c60f08a65081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:38.038553 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665660e-e22b-41ce-92d0-c60f08a65081 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:38.039201 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665660e-e22b-41ce-92d0-c60f08a65081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:38.041175 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665660e-e22b-41ce-92d0-c60f08a65081 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:38.063727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f57f98-ffd4-4e90-bd74-eaf59eff466a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:38.067906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f315324-0ebe-49c5-a482-5058b30f2ea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:38.068524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f315324-0ebe-49c5-a482-5058b30f2ea1 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:38.068800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f315324-0ebe-49c5-a482-5058b30f2ea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:38.069059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f315324-0ebe-49c5-a482-5058b30f2ea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:38.078016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 971/1936] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:38 2022] POST /identity/v3/projects => generated 365 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:38.081854 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665660e-e22b-41ce-92d0-c60f08a65081 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:38.086528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f315324-0ebe-49c5-a482-5058b30f2ea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:38.089789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f315324-0ebe-49c5-a482-5058b30f2ea1 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:38.102421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 966/1937] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:38.153682 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665660e-e22b-41ce-92d0-c60f08a65081 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:38.159468 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ea674-f490-4f29-bac2-5ab366860fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:38.160085 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ea674-f490-4f29-bac2-5ab366860fea admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:38.160362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ea674-f490-4f29-bac2-5ab366860fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:38.160629 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ea674-f490-4f29-bac2-5ab366860fea admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:38.161345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ea674-f490-4f29-bac2-5ab366860fea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:38.163364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ea674-f490-4f29-bac2-5ab366860fea admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:38.165724 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f315324-0ebe-49c5-a482-5058b30f2ea1 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:38.167407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9f315324-0ebe-49c5-a482-5058b30f2ea1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:38.181806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f315324-0ebe-49c5-a482-5058b30f2ea1 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:38.187194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:38.187736 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 972/1938] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:38 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:32:38.187846 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:38.188153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:38.188360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] PATH_INFO: `/v3/projects/a341eb72d4664b26a7d1816c00b934ff` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:38.189493 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a341eb72d4664b26a7d1816c00b934ff)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:38.192135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:38.192994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ea674-f490-4f29-bac2-5ab366860fea admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:38.205694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ea674-f490-4f29-bac2-5ab366860fea admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:38.214861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0c191-7a34-4247-a6fb-491de0870897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:38.215863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0c191-7a34-4247-a6fb-491de0870897 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:38.216347 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0c191-7a34-4247-a6fb-491de0870897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:38.217028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0c191-7a34-4247-a6fb-491de0870897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:38.217945 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0c191-7a34-4247-a6fb-491de0870897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:38.220134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0c191-7a34-4247-a6fb-491de0870897 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:38.231365 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 973/1939] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:38.236326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0c191-7a34-4247-a6fb-491de0870897 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:38.248023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0c191-7a34-4247-a6fb-491de0870897 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:38.254308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a341eb72d4664b26a7d1816c00b934ff', 'request_id': 'req-87c94176-f059-4406-8f87-891a2ac84256'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.261799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a341eb72d4664b26a7d1816c00b934ff', 'request_id': 'req-87c94176-f059-4406-8f87-891a2ac84256'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.270425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a341eb72d4664b26a7d1816c00b934ff', 'request_id': 'req-87c94176-f059-4406-8f87-891a2ac84256'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.286624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] The token cache is being invalidate because project a341eb72d4664b26a7d1816c00b934ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:38.286901 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.287715 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.290381 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 967/1940] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:38 2022] DELETE /identity/v3/projects/a341eb72d4664b26a7d1816c00b934ff => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:38.294318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:38.295862 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:38.332875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee50bf4-323e-4356-94db-7e76b852fe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:38.333871 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee50bf4-323e-4356-94db-7e76b852fe99 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:38.334422 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee50bf4-323e-4356-94db-7e76b852fe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:38.334920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee50bf4-323e-4356-94db-7e76b852fe99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:38.335650 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee50bf4-323e-4356-94db-7e76b852fe99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:38.337965 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee50bf4-323e-4356-94db-7e76b852fe99 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:38.339728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:38.356998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 974/1941] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:32:38.357847 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee50bf4-323e-4356-94db-7e76b852fe99 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:38.360239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1ee50bf4-323e-4356-94db-7e76b852fe99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:38.381602 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee50bf4-323e-4356-94db-7e76b852fe99 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:38.388203 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64be1d-dec1-4f17-9b66-4d6659fc7177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:38.388958 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64be1d-dec1-4f17-9b66-4d6659fc7177 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:38.389240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64be1d-dec1-4f17-9b66-4d6659fc7177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:38.389544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64be1d-dec1-4f17-9b66-4d6659fc7177 admin admin] PATH_INFO: `/v3/projects/201e9ee4b1d845c3aee433d572423391/users/6c2f8438ca1e46f6a1c769795a7a020e/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:38.394070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c94176-f059-4406-8f87-891a2ac84256 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:38.400121 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e917617-c655-45b3-9e6a-4e3a68febb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:38.400896 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e917617-c655-45b3-9e6a-4e3a68febb86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:38.401268 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e917617-c655-45b3-9e6a-4e3a68febb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:38.401633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e917617-c655-45b3-9e6a-4e3a68febb86 admin admin] PATH_INFO: `/v3/users/40d84f8a05f44a71977cc30c6b399d34` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:38.402958 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e917617-c655-45b3-9e6a-4e3a68febb86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40d84f8a05f44a71977cc30c6b399d34)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:38.405062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e917617-c655-45b3-9e6a-4e3a68febb86 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:38.416645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64be1d-dec1-4f17-9b66-4d6659fc7177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=201e9ee4b1d845c3aee433d572423391, user_id=6c2f8438ca1e46f6a1c769795a7a020e, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:38.419635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64be1d-dec1-4f17-9b66-4d6659fc7177 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:38.441694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:38.444415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 975/1942] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:38 2022] PUT /identity/v3/projects/201e9ee4b1d845c3aee433d572423391/users/6c2f8438ca1e46f6a1c769795a7a020e/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:38.458477 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333abab-491c-4bd4-818a-3ce41860ca7d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:38.458921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333abab-491c-4bd4-818a-3ce41860ca7d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:38.459292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333abab-491c-4bd4-818a-3ce41860ca7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:38.497246 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-3333abab-491c-4bd4-818a-3ce41860ca7d None None] MFA Rules not processed for user `6c2f8438ca1e46f6a1c769795a7a020e`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:38.501339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5e917617-c655-45b3-9e6a-4e3a68febb86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40d84f8a05f44a71977cc30c6b399d34'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.502416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5e917617-c655-45b3-9e6a-4e3a68febb86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40d84f8a05f44a71977cc30c6b399d34'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.503477 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5e917617-c655-45b3-9e6a-4e3a68febb86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40d84f8a05f44a71977cc30c6b399d34'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.506552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-3333abab-491c-4bd4-818a-3ce41860ca7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:38.519866 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5e917617-c655-45b3-9e6a-4e3a68febb86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40d84f8a05f44a71977cc30c6b399d34'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.534002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5e917617-c655-45b3-9e6a-4e3a68febb86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40d84f8a05f44a71977cc30c6b399d34'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.549471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 968/1943] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:38 2022] DELETE /identity/v3/users/40d84f8a05f44a71977cc30c6b399d34 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:38.550799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e917617-c655-45b3-9e6a-4e3a68febb86 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:38.552025 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5e917617-c655-45b3-9e6a-4e3a68febb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:38.618760 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 976/1944] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:38.623097 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333abab-491c-4bd4-818a-3ce41860ca7d None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:38.626180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-3333abab-491c-4bd4-818a-3ce41860ca7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:38.656762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e917617-c655-45b3-9e6a-4e3a68febb86 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:38.671506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:38.672174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:38.672450 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:38.672829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] PATH_INFO: `/v3/projects/0a0e68e24a3649f0bc00dab3192fdf11` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:38.675051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a0e68e24a3649f0bc00dab3192fdf11)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:38.678291 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:38.745264 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333abab-491c-4bd4-818a-3ce41860ca7d None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:38.768230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a0e68e24a3649f0bc00dab3192fdf11', 'request_id': 'req-853f1b70-bc03-4a2f-951f-5a57b2512780'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.776447 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a0e68e24a3649f0bc00dab3192fdf11', 'request_id': 'req-853f1b70-bc03-4a2f-951f-5a57b2512780'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.786454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a0e68e24a3649f0bc00dab3192fdf11', 'request_id': 'req-853f1b70-bc03-4a2f-951f-5a57b2512780'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.808764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] The token cache is being invalidate because project 0a0e68e24a3649f0bc00dab3192fdf11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:38.809336 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.810278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:38.812343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 969/1945] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:38 2022] DELETE /identity/v3/projects/0a0e68e24a3649f0bc00dab3192fdf11 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:38.815934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:38.817501 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:38.838756 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68d72c-5c8e-4809-97eb-832023f6556b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:38.842008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68d72c-5c8e-4809-97eb-832023f6556b admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:38.842287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68d72c-5c8e-4809-97eb-832023f6556b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:38.842579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68d72c-5c8e-4809-97eb-832023f6556b admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:38.843381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68d72c-5c8e-4809-97eb-832023f6556b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:38.845345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68d72c-5c8e-4809-97eb-832023f6556b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:38.870931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:38.877563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:38.911856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f1b70-bc03-4a2f-951f-5a57b2512780 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:38.912011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 977/1946] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:38 2022] POST /identity/v3/projects => generated 361 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:38.916811 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68d72c-5c8e-4809-97eb-832023f6556b admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:38.920844 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0c68d72c-5c8e-4809-97eb-832023f6556b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:39.026104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef0415-3c8e-42cf-b1f4-22ac01f898c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:39.026359 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef0415-3c8e-42cf-b1f4-22ac01f898c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:39.026684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef0415-3c8e-42cf-b1f4-22ac01f898c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:39.027111 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef0415-3c8e-42cf-b1f4-22ac01f898c5 admin admin] PATH_INFO: `/v3/users/79e00f2a48334805be57af4b6c32e6fb` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:39.028814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef0415-3c8e-42cf-b1f4-22ac01f898c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e00f2a48334805be57af4b6c32e6fb)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:39.031085 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef0415-3c8e-42cf-b1f4-22ac01f898c5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:39.034217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68d72c-5c8e-4809-97eb-832023f6556b admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:39.046045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eca535-288e-4bde-b61e-ea8c46c5ee3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:39.046753 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eca535-288e-4bde-b61e-ea8c46c5ee3f admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:39.047340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eca535-288e-4bde-b61e-ea8c46c5ee3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:39.047685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eca535-288e-4bde-b61e-ea8c46c5ee3f admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:39.049373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eca535-288e-4bde-b61e-ea8c46c5ee3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:39.052652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eca535-288e-4bde-b61e-ea8c46c5ee3f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:39.095379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 978/1947] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:38 2022] POST /identity/v3/users => generated 287 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:39.098620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eca535-288e-4bde-b61e-ea8c46c5ee3f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:39.109484 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eca535-288e-4bde-b61e-ea8c46c5ee3f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:39.112461 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eaef0415-3c8e-42cf-b1f4-22ac01f898c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e00f2a48334805be57af4b6c32e6fb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:39.113539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eaef0415-3c8e-42cf-b1f4-22ac01f898c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e00f2a48334805be57af4b6c32e6fb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:39.114669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eaef0415-3c8e-42cf-b1f4-22ac01f898c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e00f2a48334805be57af4b6c32e6fb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:39.116773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c6d5c-0a46-40a9-a97a-acf0189db78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:39.117427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c6d5c-0a46-40a9-a97a-acf0189db78c admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:39.118341 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c6d5c-0a46-40a9-a97a-acf0189db78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:39.118663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c6d5c-0a46-40a9-a97a-acf0189db78c admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:39.119310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c6d5c-0a46-40a9-a97a-acf0189db78c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:39.121840 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c6d5c-0a46-40a9-a97a-acf0189db78c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:39.123617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eaef0415-3c8e-42cf-b1f4-22ac01f898c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e00f2a48334805be57af4b6c32e6fb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:39.135827 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 979/1948] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:39.140432 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c6d5c-0a46-40a9-a97a-acf0189db78c admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:39.141200 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eaef0415-3c8e-42cf-b1f4-22ac01f898c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e00f2a48334805be57af4b6c32e6fb'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:39.142306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-457c6d5c-0a46-40a9-a97a-acf0189db78c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:39.162064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c6d5c-0a46-40a9-a97a-acf0189db78c admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:39.172943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 970/1949] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:38 2022] DELETE /identity/v3/users/79e00f2a48334805be57af4b6c32e6fb => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:39.174674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef0415-3c8e-42cf-b1f4-22ac01f898c5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:39.177279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5888b77-e450-46f5-84e3-78cf41e312d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:39.179229 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5888b77-e450-46f5-84e3-78cf41e312d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:39.179626 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-eaef0415-3c8e-42cf-b1f4-22ac01f898c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:39.180207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5888b77-e450-46f5-84e3-78cf41e312d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:39.180486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5888b77-e450-46f5-84e3-78cf41e312d3 admin admin] PATH_INFO: `/v3/projects/9350cc6931254b93ab5f424e7ec4794f/users/db8e44083d4c4ed99306cb455b3e0048/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:39.215755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5888b77-e450-46f5-84e3-78cf41e312d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9350cc6931254b93ab5f424e7ec4794f, user_id=db8e44083d4c4ed99306cb455b3e0048, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:39.220305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5888b77-e450-46f5-84e3-78cf41e312d3 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:39.261308 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 980/1950] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:39 2022] PUT /identity/v3/projects/9350cc6931254b93ab5f424e7ec4794f/users/db8e44083d4c4ed99306cb455b3e0048/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:39.262708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5888b77-e450-46f5-84e3-78cf41e312d3 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:39.370617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef0415-3c8e-42cf-b1f4-22ac01f898c5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:39.377466 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:39.425311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5888b77-e450-46f5-84e3-78cf41e312d3 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:39.434279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc052fd-fb59-4b68-9380-a332f7ffe602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:39.436517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc052fd-fb59-4b68-9380-a332f7ffe602 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:39.437039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc052fd-fb59-4b68-9380-a332f7ffe602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:39.437456 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc052fd-fb59-4b68-9380-a332f7ffe602 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:39.438784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc052fd-fb59-4b68-9380-a332f7ffe602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:39.440632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc052fd-fb59-4b68-9380-a332f7ffe602 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:39.457141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 981/1951] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:39.461034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc052fd-fb59-4b68-9380-a332f7ffe602 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:39.475913 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc052fd-fb59-4b68-9380-a332f7ffe602 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:39.484134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35480e9-3bd7-4671-b6f0-0283af356081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:39.484930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35480e9-3bd7-4671-b6f0-0283af356081 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:39.485213 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35480e9-3bd7-4671-b6f0-0283af356081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:39.485491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35480e9-3bd7-4671-b6f0-0283af356081 admin admin] PATH_INFO: `/v3/projects/9350cc6931254b93ab5f424e7ec4794f/users/db8e44083d4c4ed99306cb455b3e0048/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:39.488319 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35480e9-3bd7-4671-b6f0-0283af356081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9350cc6931254b93ab5f424e7ec4794f, user_id=db8e44083d4c4ed99306cb455b3e0048, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:39.491983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35480e9-3bd7-4671-b6f0-0283af356081 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:39.502699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ed095-3054-4643-b6f9-37828829df9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:39.503490 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ed095-3054-4643-b6f9-37828829df9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:39.503761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ed095-3054-4643-b6f9-37828829df9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:39.505367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ed095-3054-4643-b6f9-37828829df9a admin admin] PATH_INFO: `/v3/projects/a5bb9dbcb62a4de58c1c6d7d6324a8e4` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:39.511636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ed095-3054-4643-b6f9-37828829df9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5bb9dbcb62a4de58c1c6d7d6324a8e4)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:39.516166 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ed095-3054-4643-b6f9-37828829df9a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:39.523380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 982/1952] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:39 2022] PUT /identity/v3/projects/9350cc6931254b93ab5f424e7ec4794f/users/db8e44083d4c4ed99306cb455b3e0048/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:39.525000 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afbd08-2db2-43ff-95b0-558054ff20ac None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:39.525312 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afbd08-2db2-43ff-95b0-558054ff20ac None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:39.525563 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afbd08-2db2-43ff-95b0-558054ff20ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:39.569983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-02afbd08-2db2-43ff-95b0-558054ff20ac None None] MFA Rules not processed for user `db8e44083d4c4ed99306cb455b3e0048`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:39.571295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-02afbd08-2db2-43ff-95b0-558054ff20ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:39.587755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f88ed095-3054-4643-b6f9-37828829df9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5bb9dbcb62a4de58c1c6d7d6324a8e4', 'request_id': 'req-f88ed095-3054-4643-b6f9-37828829df9a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:39.595905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f88ed095-3054-4643-b6f9-37828829df9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5bb9dbcb62a4de58c1c6d7d6324a8e4', 'request_id': 'req-f88ed095-3054-4643-b6f9-37828829df9a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:39.604847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f88ed095-3054-4643-b6f9-37828829df9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5bb9dbcb62a4de58c1c6d7d6324a8e4', 'request_id': 'req-f88ed095-3054-4643-b6f9-37828829df9a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:39.614780 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:39.631430 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f88ed095-3054-4643-b6f9-37828829df9a admin admin] The token cache is being invalidate because project a5bb9dbcb62a4de58c1c6d7d6324a8e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:39.631750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f88ed095-3054-4643-b6f9-37828829df9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:39.635110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f88ed095-3054-4643-b6f9-37828829df9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:39.638372 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 971/1953] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:39 2022] DELETE /identity/v3/projects/a5bb9dbcb62a4de58c1c6d7d6324a8e4 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:39.640110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ab704-99af-4ea6-b15a-4ba4096dde5b None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:39.640439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ab704-99af-4ea6-b15a-4ba4096dde5b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:39.642150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ab704-99af-4ea6-b15a-4ba4096dde5b None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:39.642871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 972/1954] 10.222.0.23 () {54 vars in 921 bytes} [Fri Sep 9 16:32:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:32:39.644370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cfc79-b38e-4cfb-9907-948ed7bfa5a5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:39.644682 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cfc79-b38e-4cfb-9907-948ed7bfa5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:39.644949 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cfc79-b38e-4cfb-9907-948ed7bfa5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:39.672292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e31cfc79-b38e-4cfb-9907-948ed7bfa5a5 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:39.674014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e31cfc79-b38e-4cfb-9907-948ed7bfa5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:39.758026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 983/1955] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:39 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:39.762030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965c516-3c69-445e-94c3-b1bc2cf95c49 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:39.762327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965c516-3c69-445e-94c3-b1bc2cf95c49 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:39.762633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965c516-3c69-445e-94c3-b1bc2cf95c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:39.809815 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-8965c516-3c69-445e-94c3-b1bc2cf95c49 None None] MFA Rules not processed for user `152a41ad598b4b01b1881d928c53c848`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:39.812494 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8965c516-3c69-445e-94c3-b1bc2cf95c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:39.830550 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 973/1956] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:39.831113 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cfc79-b38e-4cfb-9907-948ed7bfa5a5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:39.832531 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e31cfc79-b38e-4cfb-9907-948ed7bfa5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:39.902031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:39.905422 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 984/1957] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Sep 9 16:32:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:39.908188 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b6229-c1f1-4733-9939-d429d80f006f None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:39.908834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b6229-c1f1-4733-9939-d429d80f006f None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:39.909287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b6229-c1f1-4733-9939-d429d80f006f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:39.926936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cfc79-b38e-4cfb-9907-948ed7bfa5a5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:39.933472 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef556fe-6197-48ee-9157-f6a63da8d28c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:39.934070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef556fe-6197-48ee-9157-f6a63da8d28c service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:39.934349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef556fe-6197-48ee-9157-f6a63da8d28c service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:39.934687 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef556fe-6197-48ee-9157-f6a63da8d28c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:39.938124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-8b6b6229-c1f1-4733-9939-d429d80f006f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:39.940398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8b6b6229-c1f1-4733-9939-d429d80f006f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:39.946220 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef556fe-6197-48ee-9157-f6a63da8d28c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:39.948106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef556fe-6197-48ee-9157-f6a63da8d28c service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:39.951577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 985/1958] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:39.956151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b6229-c1f1-4733-9939-d429d80f006f None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:39.962680 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 974/1959] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:39 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:39.969967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b6229-c1f1-4733-9939-d429d80f006f None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:39.976726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeebb227-df3c-4e60-81ef-27d25282da8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:39.977344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebb227-df3c-4e60-81ef-27d25282da8f admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:39.977629 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebb227-df3c-4e60-81ef-27d25282da8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:39.977901 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebb227-df3c-4e60-81ef-27d25282da8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:39.978786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeebb227-df3c-4e60-81ef-27d25282da8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:39.979053 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeebb227-df3c-4e60-81ef-27d25282da8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:39.981068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeebb227-df3c-4e60-81ef-27d25282da8f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:39.996293 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 986/1960] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:32:40.001069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef556fe-6197-48ee-9157-f6a63da8d28c service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:40.021660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef556fe-6197-48ee-9157-f6a63da8d28c service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:40.029943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c7189-b981-4d90-b89f-0ab159f8c934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:40.031087 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c7189-b981-4d90-b89f-0ab159f8c934 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:40.031754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c7189-b981-4d90-b89f-0ab159f8c934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:40.032267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c7189-b981-4d90-b89f-0ab159f8c934 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:40.033454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c7189-b981-4d90-b89f-0ab159f8c934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:40.036369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c7189-b981-4d90-b89f-0ab159f8c934 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:40.069845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeebb227-df3c-4e60-81ef-27d25282da8f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:40.081424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeebb227-df3c-4e60-81ef-27d25282da8f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:40.093496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae21f9e-abfa-4669-b2e9-43254a593dcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:40.096173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae21f9e-abfa-4669-b2e9-43254a593dcf service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:40.096590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae21f9e-abfa-4669-b2e9-43254a593dcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:40.096966 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae21f9e-abfa-4669-b2e9-43254a593dcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:40.099476 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 975/1961] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:39 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:40.103154 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7ae21f9e-abfa-4669-b2e9-43254a593dcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:40.103434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c7189-b981-4d90-b89f-0ab159f8c934 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:40.210438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c7189-b981-4d90-b89f-0ab159f8c934 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:40.214997 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae21f9e-abfa-4669-b2e9-43254a593dcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:40.216661 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae21f9e-abfa-4669-b2e9-43254a593dcf service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:40.219821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55ffd9-e8aa-43a6-a959-b7b41989c6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:40.220568 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55ffd9-e8aa-43a6-a959-b7b41989c6af admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:40.220973 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55ffd9-e8aa-43a6-a959-b7b41989c6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:40.221331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55ffd9-e8aa-43a6-a959-b7b41989c6af admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:40.222083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55ffd9-e8aa-43a6-a959-b7b41989c6af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:40.224429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55ffd9-e8aa-43a6-a959-b7b41989c6af admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:40.240104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 987/1962] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:40.254272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 976/1963] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:40 2022] POST /identity/v3/users => generated 285 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:40.258323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae21f9e-abfa-4669-b2e9-43254a593dcf service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:40.271852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae21f9e-abfa-4669-b2e9-43254a593dcf service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:40.277037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdda0a-2912-4ec7-bbb0-67c49baf7322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:40.277670 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdda0a-2912-4ec7-bbb0-67c49baf7322 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:40.277948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdda0a-2912-4ec7-bbb0-67c49baf7322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:40.278212 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdda0a-2912-4ec7-bbb0-67c49baf7322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:40.278922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbdda0a-2912-4ec7-bbb0-67c49baf7322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:40.280613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbdda0a-2912-4ec7-bbb0-67c49baf7322 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:40.292226 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 988/1964] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:40.296034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55ffd9-e8aa-43a6-a959-b7b41989c6af admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:40.310264 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55ffd9-e8aa-43a6-a959-b7b41989c6af admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:40.315901 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66c6f4-0505-422a-b7a1-dd9132a9820c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:40.316538 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66c6f4-0505-422a-b7a1-dd9132a9820c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:40.316814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66c6f4-0505-422a-b7a1-dd9132a9820c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:40.317083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66c6f4-0505-422a-b7a1-dd9132a9820c admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:40.317986 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66c6f4-0505-422a-b7a1-dd9132a9820c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:40.319807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66c6f4-0505-422a-b7a1-dd9132a9820c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:40.332493 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 977/1965] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:40.336740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdda0a-2912-4ec7-bbb0-67c49baf7322 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:40.349957 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdda0a-2912-4ec7-bbb0-67c49baf7322 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:40.355380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0210d44-9f37-4be0-af7f-0a8a2c948c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:40.356119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0210d44-9f37-4be0-af7f-0a8a2c948c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:40.356403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0210d44-9f37-4be0-af7f-0a8a2c948c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:40.357383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0210d44-9f37-4be0-af7f-0a8a2c948c81 admin admin] PATH_INFO: `/v3/projects/89efe72a8514421d84221b296f6b7f34/users/a3b657615b2643d5829600d3afb4103c/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:40.375205 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:40.379081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0210d44-9f37-4be0-af7f-0a8a2c948c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89efe72a8514421d84221b296f6b7f34, user_id=a3b657615b2643d5829600d3afb4103c, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:40.382320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0210d44-9f37-4be0-af7f-0a8a2c948c81 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:40.404677 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 989/1966] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:40 2022] PUT /identity/v3/projects/89efe72a8514421d84221b296f6b7f34/users/a3b657615b2643d5829600d3afb4103c/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:40.415773 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb2f9b-c58a-4cce-befe-704bfca1def7 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:40.416082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb2f9b-c58a-4cce-befe-704bfca1def7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:40.416404 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb2f9b-c58a-4cce-befe-704bfca1def7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:40.449191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0210d44-9f37-4be0-af7f-0a8a2c948c81 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:40.451092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0dbb2f9b-c58a-4cce-befe-704bfca1def7 None None] MFA Rules not processed for user `a3b657615b2643d5829600d3afb4103c`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:40.451524 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f0210d44-9f37-4be0-af7f-0a8a2c948c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:40.452304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0dbb2f9b-c58a-4cce-befe-704bfca1def7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:40.583013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 978/1967] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:40.585032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb2f9b-c58a-4cce-befe-704bfca1def7 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:40.586487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0dbb2f9b-c58a-4cce-befe-704bfca1def7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:40.598187 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0210d44-9f37-4be0-af7f-0a8a2c948c81 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:40.604394 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:40.605307 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:40.605747 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:40.606176 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] PATH_INFO: `/v3/projects/a2d4eb7f5b214dffa52377e04d271569` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:40.607752 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2d4eb7f5b214dffa52377e04d271569)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:40.609876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:40.643700 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:40.693600 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d4eb7f5b214dffa52377e04d271569', 'request_id': 'req-bb924418-7acd-4142-8ebd-404ae3d52b38'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:40.700671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d4eb7f5b214dffa52377e04d271569', 'request_id': 'req-bb924418-7acd-4142-8ebd-404ae3d52b38'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:40.709828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d4eb7f5b214dffa52377e04d271569', 'request_id': 'req-bb924418-7acd-4142-8ebd-404ae3d52b38'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:40.715942 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb2f9b-c58a-4cce-befe-704bfca1def7 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:40.722007 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] The token cache is being invalidate because project a2d4eb7f5b214dffa52377e04d271569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:40.722466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:40.723667 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:40.725369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 990/1968] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:40 2022] DELETE /identity/v3/projects/a2d4eb7f5b214dffa52377e04d271569 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:40.726427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:40.728370 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:40.800860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de129997-ba6a-45c7-a086-7ef797dd126a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:40.801795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de129997-ba6a-45c7-a086-7ef797dd126a service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:40.802146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de129997-ba6a-45c7-a086-7ef797dd126a service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:40.802487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de129997-ba6a-45c7-a086-7ef797dd126a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:40.804911 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-de129997-ba6a-45c7-a086-7ef797dd126a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:40.813942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb924418-7acd-4142-8ebd-404ae3d52b38 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:40.818673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fa670-9ed8-45b6-8805-5da87c39ab80 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a2fb70c9ae4f4b81ac4de513b868cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:40.819324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fa670-9ed8-45b6-8805-5da87c39ab80 service glance] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:40.819711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fa670-9ed8-45b6-8805-5da87c39ab80 service glance] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:40.819990 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fa670-9ed8-45b6-8805-5da87c39ab80 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:40.821759 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de129997-ba6a-45c7-a086-7ef797dd126a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:40.822336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a82fa670-9ed8-45b6-8805-5da87c39ab80 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:40.823402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de129997-ba6a-45c7-a086-7ef797dd126a service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:40.835873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 979/1969] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:40.844395 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:40.849216 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc634c8e-4465-475c-89b0-a5488b62c31e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:40.850060 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc634c8e-4465-475c-89b0-a5488b62c31e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:40.851406 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc634c8e-4465-475c-89b0-a5488b62c31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:40.887691 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-dc634c8e-4465-475c-89b0-a5488b62c31e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:40.888924 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dc634c8e-4465-475c-89b0-a5488b62c31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:40.918078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82fa670-9ed8-45b6-8805-5da87c39ab80 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:40.919803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82fa670-9ed8-45b6-8805-5da87c39ab80 service glance] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:40.935273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 991/1970] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:40.972276 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 980/1971] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:40.993961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319903a7-4786-4e1b-9ca3-54d2cb21c78e None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:40.994243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319903a7-4786-4e1b-9ca3-54d2cb21c78e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:40.994763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319903a7-4786-4e1b-9ca3-54d2cb21c78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:41.029887 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-319903a7-4786-4e1b-9ca3-54d2cb21c78e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:41.031896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-319903a7-4786-4e1b-9ca3-54d2cb21c78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:41.051415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 992/1972] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:41.053507 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc634c8e-4465-475c-89b0-a5488b62c31e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:41.081698 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc634c8e-4465-475c-89b0-a5488b62c31e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:41.102181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4066d-4dc2-42ab-a9d5-98ee0de855e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:41.102884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4066d-4dc2-42ab-a9d5-98ee0de855e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:41.103188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4066d-4dc2-42ab-a9d5-98ee0de855e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:41.103467 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4066d-4dc2-42ab-a9d5-98ee0de855e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:41.104134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4066d-4dc2-42ab-a9d5-98ee0de855e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:41.104444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4066d-4dc2-42ab-a9d5-98ee0de855e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:41.106077 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4066d-4dc2-42ab-a9d5-98ee0de855e4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:41.133628 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 981/1973] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:32:41.137884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319903a7-4786-4e1b-9ca3-54d2cb21c78e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:41.152144 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319903a7-4786-4e1b-9ca3-54d2cb21c78e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:41.158400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e354f-d502-4c3d-9212-6a4963f2ae97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:41.159526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e354f-d502-4c3d-9212-6a4963f2ae97 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:41.160127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e354f-d502-4c3d-9212-6a4963f2ae97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:41.160619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e354f-d502-4c3d-9212-6a4963f2ae97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:41.161769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e354f-d502-4c3d-9212-6a4963f2ae97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:41.164730 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e354f-d502-4c3d-9212-6a4963f2ae97 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:41.220608 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 993/1974] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:41 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:41.225154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4066d-4dc2-42ab-a9d5-98ee0de855e4 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:41.236350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4066d-4dc2-42ab-a9d5-98ee0de855e4 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:41.296866 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:41.349772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea82f6-e326-4fea-8bdc-e401a6c9a365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:41.350545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea82f6-e326-4fea-8bdc-e401a6c9a365 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:41.351439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea82f6-e326-4fea-8bdc-e401a6c9a365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:41.351748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea82f6-e326-4fea-8bdc-e401a6c9a365 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:41.352407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea82f6-e326-4fea-8bdc-e401a6c9a365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:41.354803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea82f6-e326-4fea-8bdc-e401a6c9a365 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:41.391196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 982/1975] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:41 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:41.394452 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e354f-d502-4c3d-9212-6a4963f2ae97 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:41.412828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e354f-d502-4c3d-9212-6a4963f2ae97 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:41.425147 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d1c09-1f0a-4d1d-9bea-f3578b79819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:41.425655 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d1c09-1f0a-4d1d-9bea-f3578b79819a admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:41.426046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d1c09-1f0a-4d1d-9bea-f3578b79819a admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:41.426209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d1c09-1f0a-4d1d-9bea-f3578b79819a admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:41.427006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d1c09-1f0a-4d1d-9bea-f3578b79819a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:41.429054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d1c09-1f0a-4d1d-9bea-f3578b79819a admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:41.447648 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 994/1976] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:32:41.449288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea82f6-e326-4fea-8bdc-e401a6c9a365 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:41.463767 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea82f6-e326-4fea-8bdc-e401a6c9a365 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:41.476877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9661451-57ba-4517-9572-ba33d383bcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:41.478056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9661451-57ba-4517-9572-ba33d383bcf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:41.479178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9661451-57ba-4517-9572-ba33d383bcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:41.479623 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9661451-57ba-4517-9572-ba33d383bcf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:41.480652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9661451-57ba-4517-9572-ba33d383bcf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:41.486223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9661451-57ba-4517-9572-ba33d383bcf8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:41.503800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 983/1977] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:41.505288 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d1c09-1f0a-4d1d-9bea-f3578b79819a admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:41.519837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d1c09-1f0a-4d1d-9bea-f3578b79819a admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:41.526630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f388a-1e59-4a8f-b017-7f9ea124c09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:41.527323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f388a-1e59-4a8f-b017-7f9ea124c09b admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:41.527631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f388a-1e59-4a8f-b017-7f9ea124c09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:41.527904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f388a-1e59-4a8f-b017-7f9ea124c09b admin admin] PATH_INFO: `/v3/projects/abb443b8dc364b16ad535e840b5bc8cd/users/2ddf4cfa095440f6a5b46c382556843f/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:41.547325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f388a-1e59-4a8f-b017-7f9ea124c09b admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb443b8dc364b16ad535e840b5bc8cd, user_id=2ddf4cfa095440f6a5b46c382556843f, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:41.551254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f388a-1e59-4a8f-b017-7f9ea124c09b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:41.598606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 995/1978] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:41 2022] PUT /identity/v3/projects/abb443b8dc364b16ad535e840b5bc8cd/users/2ddf4cfa095440f6a5b46c382556843f/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:41.601740 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b1884-4fde-4ea4-9a0d-9f0fcac3ea16 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:41.602514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b1884-4fde-4ea4-9a0d-9f0fcac3ea16 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:41.603094 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b1884-4fde-4ea4-9a0d-9f0fcac3ea16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:41.654144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7b2b1884-4fde-4ea4-9a0d-9f0fcac3ea16 None None] MFA Rules not processed for user `2ddf4cfa095440f6a5b46c382556843f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:41.656793 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7b2b1884-4fde-4ea4-9a0d-9f0fcac3ea16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:41.799019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 984/1979] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:41.848116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f388a-1e59-4a8f-b017-7f9ea124c09b admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:41.851097 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-601f388a-1e59-4a8f-b017-7f9ea124c09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:41.960863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f388a-1e59-4a8f-b017-7f9ea124c09b admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:41.964809 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdfe1a-e81e-4fdf-a9cd-1965bd07cf53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:41.965439 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdfe1a-e81e-4fdf-a9cd-1965bd07cf53 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:41.965717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdfe1a-e81e-4fdf-a9cd-1965bd07cf53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:41.965988 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdfe1a-e81e-4fdf-a9cd-1965bd07cf53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:41.983719 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdfe1a-e81e-4fdf-a9cd-1965bd07cf53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:41.983719 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdfe1a-e81e-4fdf-a9cd-1965bd07cf53 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:41.989363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 996/1980] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:43.381497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b1884-4fde-4ea4-9a0d-9f0fcac3ea16 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:43.384782 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7b2b1884-4fde-4ea4-9a0d-9f0fcac3ea16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:43.489510 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b1884-4fde-4ea4-9a0d-9f0fcac3ea16 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:43.494129 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b60ac8-b07a-4e50-9e50-a935cd2547f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:43.495019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b60ac8-b07a-4e50-9e50-a935cd2547f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:43.495217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b60ac8-b07a-4e50-9e50-a935cd2547f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:43.495496 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b60ac8-b07a-4e50-9e50-a935cd2547f1 admin admin] PATH_INFO: `/v3/users/b56d909c22244d7d80f6c366019b4029` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:43.496638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b60ac8-b07a-4e50-9e50-a935cd2547f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b56d909c22244d7d80f6c366019b4029)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:43.500931 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b60ac8-b07a-4e50-9e50-a935cd2547f1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:43.525135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdfe1a-e81e-4fdf-a9cd-1965bd07cf53 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:43.528399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-eafdfe1a-e81e-4fdf-a9cd-1965bd07cf53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:43.546316 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdfe1a-e81e-4fdf-a9cd-1965bd07cf53 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:43.554254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff9b98-0e4d-4ed7-b261-766813c95b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:43.555193 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff9b98-0e4d-4ed7-b261-766813c95b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:43.555671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff9b98-0e4d-4ed7-b261-766813c95b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:43.555970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff9b98-0e4d-4ed7-b261-766813c95b4f admin admin] PATH_INFO: `/v3/users/6c2f8438ca1e46f6a1c769795a7a020e` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:43.557885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff9b98-0e4d-4ed7-b261-766813c95b4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c2f8438ca1e46f6a1c769795a7a020e)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:43.561059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff9b98-0e4d-4ed7-b261-766813c95b4f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:43.561245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d4b60ac8-b07a-4e50-9e50-a935cd2547f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b56d909c22244d7d80f6c366019b4029'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:43.562080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d4b60ac8-b07a-4e50-9e50-a935cd2547f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b56d909c22244d7d80f6c366019b4029'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:43.563436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d4b60ac8-b07a-4e50-9e50-a935cd2547f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b56d909c22244d7d80f6c366019b4029'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:43.565589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:43.584170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d4b60ac8-b07a-4e50-9e50-a935cd2547f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b56d909c22244d7d80f6c366019b4029'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:43.603586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d4b60ac8-b07a-4e50-9e50-a935cd2547f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b56d909c22244d7d80f6c366019b4029'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:43.623011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 985/1981] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:43 2022] DELETE /identity/v3/users/b56d909c22244d7d80f6c366019b4029 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:43.627611 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1263db-1b04-4b64-a9fd-cc9b5b41c5ec None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:43.628157 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1263db-1b04-4b64-a9fd-cc9b5b41c5ec None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:43.628542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1263db-1b04-4b64-a9fd-cc9b5b41c5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:43.631869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-b8ff9b98-0e4d-4ed7-b261-766813c95b4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c2f8438ca1e46f6a1c769795a7a020e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:43.632677 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-b8ff9b98-0e4d-4ed7-b261-766813c95b4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c2f8438ca1e46f6a1c769795a7a020e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:43.633368 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-b8ff9b98-0e4d-4ed7-b261-766813c95b4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c2f8438ca1e46f6a1c769795a7a020e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:43.635813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:43.647171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-b8ff9b98-0e4d-4ed7-b261-766813c95b4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c2f8438ca1e46f6a1c769795a7a020e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:43.660676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-6a1263db-1b04-4b64-a9fd-cc9b5b41c5ec None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:43.662295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6a1263db-1b04-4b64-a9fd-cc9b5b41c5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:43.670383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-b8ff9b98-0e4d-4ed7-b261-766813c95b4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c2f8438ca1e46f6a1c769795a7a020e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:43.693105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 997/1982] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:43 2022] DELETE /identity/v3/users/6c2f8438ca1e46f6a1c769795a7a020e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:43.694648 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673011e-76df-4a6f-b024-c1055d5e5e62 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:43.694934 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673011e-76df-4a6f-b024-c1055d5e5e62 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:43.695205 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673011e-76df-4a6f-b024-c1055d5e5e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:43.732734 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-1673011e-76df-4a6f-b024-c1055d5e5e62 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:43.734159 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1673011e-76df-4a6f-b024-c1055d5e5e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:43.767283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 986/1983] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:43.778829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1263db-1b04-4b64-a9fd-cc9b5b41c5ec None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:43.780798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6a1263db-1b04-4b64-a9fd-cc9b5b41c5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:43.818846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 998/1984] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:43.831037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673011e-76df-4a6f-b024-c1055d5e5e62 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:43.833397 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1673011e-76df-4a6f-b024-c1055d5e5e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:43.896270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1263db-1b04-4b64-a9fd-cc9b5b41c5ec None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:43.903460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673011e-76df-4a6f-b024-c1055d5e5e62 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:43.906421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b948e2-8dad-4092-b644-941891e3222e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:43.906421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b948e2-8dad-4092-b644-941891e3222e service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:43.906830 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b948e2-8dad-4092-b644-941891e3222e service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:43.910154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b948e2-8dad-4092-b644-941891e3222e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:43.918140 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0bcd8-9d20-4f11-b1a9-8a4b074457d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:43.919075 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0bcd8-9d20-4f11-b1a9-8a4b074457d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:43.919465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0bcd8-9d20-4f11-b1a9-8a4b074457d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:43.919773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0bcd8-9d20-4f11-b1a9-8a4b074457d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:43.927320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b948e2-8dad-4092-b644-941891e3222e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:43.929377 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b948e2-8dad-4092-b644-941891e3222e service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:43.940329 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0bcd8-9d20-4f11-b1a9-8a4b074457d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:43.941681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0bcd8-9d20-4f11-b1a9-8a4b074457d4 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:43.945357 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 987/1985] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:43.973261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 999/1986] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:44.388679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b948e2-8dad-4092-b644-941891e3222e service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:44.393901 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f9b948e2-8dad-4092-b644-941891e3222e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:44.409798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b948e2-8dad-4092-b644-941891e3222e service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:44.421988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:44.422722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:44.423002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:44.423271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] PATH_INFO: `/v3/projects/201e9ee4b1d845c3aee433d572423391` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:44.424896 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] RBAC: Authorizing `identity:delete_project(project_id=201e9ee4b1d845c3aee433d572423391)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:44.427093 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:44.486783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201e9ee4b1d845c3aee433d572423391', 'request_id': 'req-c542c375-ad8e-449a-9aa7-24451231818f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.493335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201e9ee4b1d845c3aee433d572423391', 'request_id': 'req-c542c375-ad8e-449a-9aa7-24451231818f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.503639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201e9ee4b1d845c3aee433d572423391', 'request_id': 'req-c542c375-ad8e-449a-9aa7-24451231818f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.507038 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:44.523554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] The token cache is being invalidate because project 201e9ee4b1d845c3aee433d572423391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:44.524184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.525354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.527755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 988/1987] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:44 2022] DELETE /identity/v3/projects/201e9ee4b1d845c3aee433d572423391 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:44.532698 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0bcd8-9d20-4f11-b1a9-8a4b074457d4 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:44.548222 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:44.550463 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:44.571114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0bcd8-9d20-4f11-b1a9-8a4b074457d4 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:44.661191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f1f9c-bb70-4b54-87cf-59d1297d5c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:44.662070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f1f9c-bb70-4b54-87cf-59d1297d5c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:44.662560 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f1f9c-bb70-4b54-87cf-59d1297d5c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:44.662981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f1f9c-bb70-4b54-87cf-59d1297d5c64 admin admin] PATH_INFO: `/v3/users/db8e44083d4c4ed99306cb455b3e0048` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:44.664311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f1f9c-bb70-4b54-87cf-59d1297d5c64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db8e44083d4c4ed99306cb455b3e0048)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:44.667400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f1f9c-bb70-4b54-87cf-59d1297d5c64 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:44.669923 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542c375-ad8e-449a-9aa7-24451231818f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:44.676310 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a119b8e-789c-4d0d-9340-44c8bf02905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:44.677036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a119b8e-789c-4d0d-9340-44c8bf02905b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:44.677309 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a119b8e-789c-4d0d-9340-44c8bf02905b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:44.677718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a119b8e-789c-4d0d-9340-44c8bf02905b admin admin] PATH_INFO: `/v3/projects/caaf15310a3e45a6b8febb4566e72d79` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:44.678912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a119b8e-789c-4d0d-9340-44c8bf02905b admin admin] RBAC: Authorizing `identity:delete_project(project_id=caaf15310a3e45a6b8febb4566e72d79)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:44.681156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a119b8e-789c-4d0d-9340-44c8bf02905b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:44.682767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:44.733912 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-592f1f9c-bb70-4b54-87cf-59d1297d5c64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db8e44083d4c4ed99306cb455b3e0048'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.734613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-592f1f9c-bb70-4b54-87cf-59d1297d5c64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db8e44083d4c4ed99306cb455b3e0048'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.735256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-592f1f9c-bb70-4b54-87cf-59d1297d5c64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db8e44083d4c4ed99306cb455b3e0048'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.738555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a119b8e-789c-4d0d-9340-44c8bf02905b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caaf15310a3e45a6b8febb4566e72d79', 'request_id': 'req-6a119b8e-789c-4d0d-9340-44c8bf02905b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.743149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-592f1f9c-bb70-4b54-87cf-59d1297d5c64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db8e44083d4c4ed99306cb455b3e0048'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.745807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a119b8e-789c-4d0d-9340-44c8bf02905b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caaf15310a3e45a6b8febb4566e72d79', 'request_id': 'req-6a119b8e-789c-4d0d-9340-44c8bf02905b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.752263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a119b8e-789c-4d0d-9340-44c8bf02905b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caaf15310a3e45a6b8febb4566e72d79', 'request_id': 'req-6a119b8e-789c-4d0d-9340-44c8bf02905b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.760874 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-592f1f9c-bb70-4b54-87cf-59d1297d5c64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db8e44083d4c4ed99306cb455b3e0048'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.778960 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a119b8e-789c-4d0d-9340-44c8bf02905b admin admin] The token cache is being invalidate because project caaf15310a3e45a6b8febb4566e72d79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:44.779254 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a119b8e-789c-4d0d-9340-44c8bf02905b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.785860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6a119b8e-789c-4d0d-9340-44c8bf02905b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:44.797298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 989/1988] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:44 2022] DELETE /identity/v3/projects/caaf15310a3e45a6b8febb4566e72d79 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:44.797462 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1000/1989] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:44 2022] DELETE /identity/v3/users/db8e44083d4c4ed99306cb455b3e0048 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:44.809569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904f6e3-332d-47ce-8964-b149ca27cb4a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:44.812198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904f6e3-332d-47ce-8964-b149ca27cb4a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:44.812578 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904f6e3-332d-47ce-8964-b149ca27cb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:44.854163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7904f6e3-332d-47ce-8964-b149ca27cb4a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:44.855452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7904f6e3-332d-47ce-8964-b149ca27cb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:44.942676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 990/1990] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:44.950562 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0e518-9b68-48a9-be82-e50bfe1f8472 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:44.950854 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0e518-9b68-48a9-be82-e50bfe1f8472 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:44.951388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0e518-9b68-48a9-be82-e50bfe1f8472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:44.971718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-98c0e518-9b68-48a9-be82-e50bfe1f8472 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:44.973145 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-98c0e518-9b68-48a9-be82-e50bfe1f8472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:44.985847 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1001/1991] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:44.989591 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904f6e3-332d-47ce-8964-b149ca27cb4a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:45.000897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904f6e3-332d-47ce-8964-b149ca27cb4a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:45.007635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e70f07-74ca-41ad-ae44-0067efe8531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:45.008263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e70f07-74ca-41ad-ae44-0067efe8531f admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.008536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e70f07-74ca-41ad-ae44-0067efe8531f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.008857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e70f07-74ca-41ad-ae44-0067efe8531f admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.009512 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e70f07-74ca-41ad-ae44-0067efe8531f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:45.009950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e70f07-74ca-41ad-ae44-0067efe8531f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:45.011613 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e70f07-74ca-41ad-ae44-0067efe8531f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:45.025047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 991/1992] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:32:45.030252 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0e518-9b68-48a9-be82-e50bfe1f8472 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:45.043260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0e518-9b68-48a9-be82-e50bfe1f8472 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:45.049040 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e4d4f-abbf-4032-bd62-ab6741ef9fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:45.050552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e4d4f-abbf-4032-bd62-ab6741ef9fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.050552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e4d4f-abbf-4032-bd62-ab6741ef9fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.050552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e4d4f-abbf-4032-bd62-ab6741ef9fd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.050992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e4d4f-abbf-4032-bd62-ab6741ef9fd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:45.053231 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e4d4f-abbf-4032-bd62-ab6741ef9fd1 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:45.094531 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1002/1993] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:45 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:45.098941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e70f07-74ca-41ad-ae44-0067efe8531f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:45.109760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e70f07-74ca-41ad-ae44-0067efe8531f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:45.115119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:45.176605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e4d4f-abbf-4032-bd62-ab6741ef9fd1 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:45.178842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-792e4d4f-abbf-4032-bd62-ab6741ef9fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:45.205547 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa18a80-dac4-4929-bfe0-ff51dd800318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:45.205919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa18a80-dac4-4929-bfe0-ff51dd800318 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.206194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa18a80-dac4-4929-bfe0-ff51dd800318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.206462 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa18a80-dac4-4929-bfe0-ff51dd800318 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.207274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa18a80-dac4-4929-bfe0-ff51dd800318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:45.209289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa18a80-dac4-4929-bfe0-ff51dd800318 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:45.242113 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 992/1994] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:45 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:45.243858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa18a80-dac4-4929-bfe0-ff51dd800318 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:45.254297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa18a80-dac4-4929-bfe0-ff51dd800318 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:45.259927 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d55046-d0ee-4831-9524-efccf83f5535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:45.260708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d55046-d0ee-4831-9524-efccf83f5535 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.260900 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e4d4f-abbf-4032-bd62-ab6741ef9fd1 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:45.261036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d55046-d0ee-4831-9524-efccf83f5535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.261248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d55046-d0ee-4831-9524-efccf83f5535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.261895 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d55046-d0ee-4831-9524-efccf83f5535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:45.263586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d55046-d0ee-4831-9524-efccf83f5535 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:45.265821 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:45.266250 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.266546 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.266817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f admin admin] PATH_INFO: `/v3/projects/9350cc6931254b93ab5f424e7ec4794f` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.267929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9350cc6931254b93ab5f424e7ec4794f)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:45.270046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:45.281722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 993/1995] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:45.282899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d55046-d0ee-4831-9524-efccf83f5535 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:45.293309 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d55046-d0ee-4831-9524-efccf83f5535 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:45.299040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8f3a9-e92c-4c60-bede-40cda6638853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:45.299656 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8f3a9-e92c-4c60-bede-40cda6638853 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.299930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8f3a9-e92c-4c60-bede-40cda6638853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.300201 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8f3a9-e92c-4c60-bede-40cda6638853 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.300848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8f3a9-e92c-4c60-bede-40cda6638853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:45.302816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8f3a9-e92c-4c60-bede-40cda6638853 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:45.318543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 994/1996] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:45.320285 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8f3a9-e92c-4c60-bede-40cda6638853 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:45.332451 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8f3a9-e92c-4c60-bede-40cda6638853 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:45.334215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9350cc6931254b93ab5f424e7ec4794f', 'request_id': 'req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:45.341342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9350cc6931254b93ab5f424e7ec4794f', 'request_id': 'req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:45.356592 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9350cc6931254b93ab5f424e7ec4794f', 'request_id': 'req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:45.359478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:45.375382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f admin admin] The token cache is being invalidate because project 9350cc6931254b93ab5f424e7ec4794f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:45.376103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:45.376485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ffc953e1-229b-423d-9a9b-e2ce1fa5bf4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:45.379026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1003/1997] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:45 2022] DELETE /identity/v3/projects/9350cc6931254b93ab5f424e7ec4794f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:45.383827 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f950dc6-5389-435e-baa8-668fb952894f None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.384173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f950dc6-5389-435e-baa8-668fb952894f None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.384479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f950dc6-5389-435e-baa8-668fb952894f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.401493 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09a2c0-49f0-4380-95f4-fda0bb6418f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:45.402204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09a2c0-49f0-4380-95f4-fda0bb6418f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.402475 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09a2c0-49f0-4380-95f4-fda0bb6418f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.402784 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09a2c0-49f0-4380-95f4-fda0bb6418f3 admin admin] PATH_INFO: `/v3/projects/e9243cdc25d5444d847db4f59c05fa6a/users/6d2fbc3668cf4a82b8eea0b67b87ca80/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.413349 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-0f950dc6-5389-435e-baa8-668fb952894f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:45.414652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0f950dc6-5389-435e-baa8-668fb952894f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:45.420550 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09a2c0-49f0-4380-95f4-fda0bb6418f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9243cdc25d5444d847db4f59c05fa6a, user_id=6d2fbc3668cf4a82b8eea0b67b87ca80, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:45.423095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1004/1998] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:45.423703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09a2c0-49f0-4380-95f4-fda0bb6418f3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:45.432169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776343c-1f71-473d-96e4-be3dffe5b438 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.432466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776343c-1f71-473d-96e4-be3dffe5b438 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.432779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776343c-1f71-473d-96e4-be3dffe5b438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.443845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 995/1999] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:45 2022] PUT /identity/v3/projects/e9243cdc25d5444d847db4f59c05fa6a/users/6d2fbc3668cf4a82b8eea0b67b87ca80/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:45.449325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da749f24-8888-4771-9ab1-37d2b260510f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.449610 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da749f24-8888-4771-9ab1-37d2b260510f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.449888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da749f24-8888-4771-9ab1-37d2b260510f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.456705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-0776343c-1f71-473d-96e4-be3dffe5b438 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:45.459820 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0776343c-1f71-473d-96e4-be3dffe5b438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:45.484904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-da749f24-8888-4771-9ab1-37d2b260510f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:45.486266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-da749f24-8888-4771-9ab1-37d2b260510f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:45.533058 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1005/2000] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:45.536829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776343c-1f71-473d-96e4-be3dffe5b438 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:45.547039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776343c-1f71-473d-96e4-be3dffe5b438 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:45.554013 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a33236-8c86-400f-9f00-4c3e4a30ae46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:45.554676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a33236-8c86-400f-9f00-4c3e4a30ae46 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.554955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a33236-8c86-400f-9f00-4c3e4a30ae46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.555225 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a33236-8c86-400f-9f00-4c3e4a30ae46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.555873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a33236-8c86-400f-9f00-4c3e4a30ae46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:45.556171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a33236-8c86-400f-9f00-4c3e4a30ae46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:45.557768 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a33236-8c86-400f-9f00-4c3e4a30ae46 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:45.569727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:45.574942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1006/2001] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:32:45.576030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a33236-8c86-400f-9f00-4c3e4a30ae46 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:45.588576 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 996/2002] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:45.594826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a33236-8c86-400f-9f00-4c3e4a30ae46 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:45.600195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da749f24-8888-4771-9ab1-37d2b260510f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:45.601540 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-da749f24-8888-4771-9ab1-37d2b260510f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:45.603195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c11d5-df5e-4ae3-9e96-1d3f3331529b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:45.604071 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c11d5-df5e-4ae3-9e96-1d3f3331529b admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.604614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c11d5-df5e-4ae3-9e96-1d3f3331529b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.605064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c11d5-df5e-4ae3-9e96-1d3f3331529b admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.605957 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c11d5-df5e-4ae3-9e96-1d3f3331529b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:45.608082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c11d5-df5e-4ae3-9e96-1d3f3331529b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:45.640671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1007/2003] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:45 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:45.644993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c11d5-df5e-4ae3-9e96-1d3f3331529b admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:45.680994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da749f24-8888-4771-9ab1-37d2b260510f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:45.719407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c11d5-df5e-4ae3-9e96-1d3f3331529b admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:45.725729 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f132b-2983-41fa-a716-e41ba1d517a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:45.726635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f132b-2983-41fa-a716-e41ba1d517a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.726920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f132b-2983-41fa-a716-e41ba1d517a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.727207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f132b-2983-41fa-a716-e41ba1d517a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.727893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f132b-2983-41fa-a716-e41ba1d517a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:45.729788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f132b-2983-41fa-a716-e41ba1d517a8 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:45.733607 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc74988-86b7-4584-a5b4-fc8cdee524ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:45.734238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc74988-86b7-4584-a5b4-fc8cdee524ce service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.734534 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc74988-86b7-4584-a5b4-fc8cdee524ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.734812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc74988-86b7-4584-a5b4-fc8cdee524ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.745702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc74988-86b7-4584-a5b4-fc8cdee524ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:45.747331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc74988-86b7-4584-a5b4-fc8cdee524ce service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:45.758014 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1008/2004] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:45 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:32:45.759262 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f132b-2983-41fa-a716-e41ba1d517a8 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:45.763419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 997/2005] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:45.770046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f132b-2983-41fa-a716-e41ba1d517a8 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:45.775142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e307c4f-22fa-4f64-b6af-aef9278d5482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:45.775795 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e307c4f-22fa-4f64-b6af-aef9278d5482 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.776117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e307c4f-22fa-4f64-b6af-aef9278d5482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.776394 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e307c4f-22fa-4f64-b6af-aef9278d5482 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.777056 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e307c4f-22fa-4f64-b6af-aef9278d5482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:45.779295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e307c4f-22fa-4f64-b6af-aef9278d5482 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:45.795527 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1009/2006] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:45.808399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc74988-86b7-4584-a5b4-fc8cdee524ce service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:45.823058 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc74988-86b7-4584-a5b4-fc8cdee524ce service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:45.830916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3da14d-02b2-4912-a75f-0cde71684d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:45.831919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3da14d-02b2-4912-a75f-0cde71684d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.832382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3da14d-02b2-4912-a75f-0cde71684d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.833047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3da14d-02b2-4912-a75f-0cde71684d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.834520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3da14d-02b2-4912-a75f-0cde71684d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:45.836429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3da14d-02b2-4912-a75f-0cde71684d62 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:45.853039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 998/2007] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:45.855438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e307c4f-22fa-4f64-b6af-aef9278d5482 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:45.868440 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e307c4f-22fa-4f64-b6af-aef9278d5482 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:45.874939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e51e9d-4fe6-4ff7-a0ac-467218ff3b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:45.875638 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e51e9d-4fe6-4ff7-a0ac-467218ff3b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.876264 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e51e9d-4fe6-4ff7-a0ac-467218ff3b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.876586 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e51e9d-4fe6-4ff7-a0ac-467218ff3b17 admin admin] PATH_INFO: `/v3/projects/50043987841c460d96d1bcb9c7aa8fad/users/cec53b13ff744d81ace451af835cf7ad/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.904204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e51e9d-4fe6-4ff7-a0ac-467218ff3b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50043987841c460d96d1bcb9c7aa8fad, user_id=cec53b13ff744d81ace451af835cf7ad, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:45.908326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e51e9d-4fe6-4ff7-a0ac-467218ff3b17 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:45.929276 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1010/2008] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:45 2022] PUT /identity/v3/projects/50043987841c460d96d1bcb9c7aa8fad/users/cec53b13ff744d81ace451af835cf7ad/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:45.941146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098965d-87eb-46fe-912d-6cf8e1c43e16 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:45.941434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098965d-87eb-46fe-912d-6cf8e1c43e16 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:45.941706 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098965d-87eb-46fe-912d-6cf8e1c43e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:45.976389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-6098965d-87eb-46fe-912d-6cf8e1c43e16 None None] MFA Rules not processed for user `cec53b13ff744d81ace451af835cf7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:45.977708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6098965d-87eb-46fe-912d-6cf8e1c43e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:46.056406 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 999/2009] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:46.060735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e51e9d-4fe6-4ff7-a0ac-467218ff3b17 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:46.067387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:46.143547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e51e9d-4fe6-4ff7-a0ac-467218ff3b17 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:46.149763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240d65b-10c5-4848-8fef-eaa9aeba6d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:46.150421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240d65b-10c5-4848-8fef-eaa9aeba6d23 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:46.151010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240d65b-10c5-4848-8fef-eaa9aeba6d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:46.151313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240d65b-10c5-4848-8fef-eaa9aeba6d23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:46.151972 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240d65b-10c5-4848-8fef-eaa9aeba6d23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:46.153908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240d65b-10c5-4848-8fef-eaa9aeba6d23 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:46.190237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1011/2010] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:46 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:46.194759 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098965d-87eb-46fe-912d-6cf8e1c43e16 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:46.214091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098965d-87eb-46fe-912d-6cf8e1c43e16 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:46.273980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:46.331018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398385f-8a6f-451a-87f2-b168dcd0f382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:46.346298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398385f-8a6f-451a-87f2-b168dcd0f382 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:46.346298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398385f-8a6f-451a-87f2-b168dcd0f382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:46.346298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398385f-8a6f-451a-87f2-b168dcd0f382 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:46.346298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398385f-8a6f-451a-87f2-b168dcd0f382 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:46.346298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398385f-8a6f-451a-87f2-b168dcd0f382 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:46.371168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1000/2011] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:46 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:46.375332 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240d65b-10c5-4848-8fef-eaa9aeba6d23 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:46.390637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240d65b-10c5-4848-8fef-eaa9aeba6d23 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:46.396476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db340d-63f7-4c0b-a269-d872185a65e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:46.397231 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db340d-63f7-4c0b-a269-d872185a65e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:46.397582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db340d-63f7-4c0b-a269-d872185a65e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:46.397924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db340d-63f7-4c0b-a269-d872185a65e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:46.398683 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db340d-63f7-4c0b-a269-d872185a65e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:46.400427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db340d-63f7-4c0b-a269-d872185a65e2 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:46.412529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1012/2012] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:46.416709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398385f-8a6f-451a-87f2-b168dcd0f382 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:46.427584 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398385f-8a6f-451a-87f2-b168dcd0f382 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:46.433289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1cb5e-73fa-4def-8074-cddea52a5260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:46.434177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1cb5e-73fa-4def-8074-cddea52a5260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:46.434560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1cb5e-73fa-4def-8074-cddea52a5260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:46.434905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1cb5e-73fa-4def-8074-cddea52a5260 admin admin] PATH_INFO: `/v3/projects/5bf63c210c9148d1902752864204e707/users/6f299fc102ef4a2f9ed92ea2fc76f93b/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:46.457519 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1cb5e-73fa-4def-8074-cddea52a5260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf63c210c9148d1902752864204e707, user_id=6f299fc102ef4a2f9ed92ea2fc76f93b, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:46.460638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1cb5e-73fa-4def-8074-cddea52a5260 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:46.475850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1001/2013] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:46 2022] PUT /identity/v3/projects/5bf63c210c9148d1902752864204e707/users/6f299fc102ef4a2f9ed92ea2fc76f93b/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:46.480247 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db340d-63f7-4c0b-a269-d872185a65e2 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:46.501795 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db340d-63f7-4c0b-a269-d872185a65e2 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:46.601864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452c399-b3ee-40fb-9af5-bb97ed3e48d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:46.602679 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452c399-b3ee-40fb-9af5-bb97ed3e48d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:46.603513 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452c399-b3ee-40fb-9af5-bb97ed3e48d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:46.603788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452c399-b3ee-40fb-9af5-bb97ed3e48d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:46.604481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452c399-b3ee-40fb-9af5-bb97ed3e48d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:46.606124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452c399-b3ee-40fb-9af5-bb97ed3e48d9 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:46.617470 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1013/2014] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:46.622426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1cb5e-73fa-4def-8074-cddea52a5260 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:46.634359 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1cb5e-73fa-4def-8074-cddea52a5260 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:46.643602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed196ff7-c5a3-483f-9484-ca37b077187e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:46.646047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed196ff7-c5a3-483f-9484-ca37b077187e admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:46.647173 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed196ff7-c5a3-483f-9484-ca37b077187e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:46.647600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed196ff7-c5a3-483f-9484-ca37b077187e admin admin] PATH_INFO: `/v3/projects/5bf63c210c9148d1902752864204e707/users/6f299fc102ef4a2f9ed92ea2fc76f93b/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:46.651973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d02588-0033-4330-9ffa-b50e57b84db9 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:46.652119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed196ff7-c5a3-483f-9484-ca37b077187e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf63c210c9148d1902752864204e707, user_id=6f299fc102ef4a2f9ed92ea2fc76f93b, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:46.652240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d02588-0033-4330-9ffa-b50e57b84db9 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:46.652428 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d02588-0033-4330-9ffa-b50e57b84db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:46.657012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed196ff7-c5a3-483f-9484-ca37b077187e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:46.690537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-48d02588-0033-4330-9ffa-b50e57b84db9 None None] MFA Rules not processed for user `6d2fbc3668cf4a82b8eea0b67b87ca80`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:46.691528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-48d02588-0033-4330-9ffa-b50e57b84db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:46.691741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1002/2015] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:46 2022] PUT /identity/v3/projects/5bf63c210c9148d1902752864204e707/users/6f299fc102ef4a2f9ed92ea2fc76f93b/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:46.693750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e58f69-39c6-401a-9398-a6d855d1bf48 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:46.694216 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e58f69-39c6-401a-9398-a6d855d1bf48 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:46.694619 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e58f69-39c6-401a-9398-a6d855d1bf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:46.741224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-08e58f69-39c6-401a-9398-a6d855d1bf48 None None] MFA Rules not processed for user `6f299fc102ef4a2f9ed92ea2fc76f93b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:46.742738 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-08e58f69-39c6-401a-9398-a6d855d1bf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:46.782326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1014/2016] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:46.794593 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d02588-0033-4330-9ffa-b50e57b84db9 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:46.796296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-48d02588-0033-4330-9ffa-b50e57b84db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:46.803885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:46.896346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d02588-0033-4330-9ffa-b50e57b84db9 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:46.900653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432b498-c464-4fee-b003-79ec8d5bac98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:46.901412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432b498-c464-4fee-b003-79ec8d5bac98 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:46.901781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432b498-c464-4fee-b003-79ec8d5bac98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:46.902093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432b498-c464-4fee-b003-79ec8d5bac98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:46.909320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1003/2017] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:46.916917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432b498-c464-4fee-b003-79ec8d5bac98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:46.918369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432b498-c464-4fee-b003-79ec8d5bac98 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:46.923913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e58f69-39c6-401a-9398-a6d855d1bf48 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:46.926065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-08e58f69-39c6-401a-9398-a6d855d1bf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:46.929827 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1015/2018] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:46.952958 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e58f69-39c6-401a-9398-a6d855d1bf48 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:46.956804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810901ab-a83f-42af-af88-044c80ce8004 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:46.957429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810901ab-a83f-42af-af88-044c80ce8004 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:46.957720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810901ab-a83f-42af-af88-044c80ce8004 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:46.958014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810901ab-a83f-42af-af88-044c80ce8004 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:46.967473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810901ab-a83f-42af-af88-044c80ce8004 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:46.968909 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810901ab-a83f-42af-af88-044c80ce8004 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:46.979530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1004/2019] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:47.009633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432b498-c464-4fee-b003-79ec8d5bac98 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:47.117335 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432b498-c464-4fee-b003-79ec8d5bac98 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:47.123232 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50cda0-73f1-4eb7-8a8c-7603a073da5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:47.123926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50cda0-73f1-4eb7-8a8c-7603a073da5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:47.124217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50cda0-73f1-4eb7-8a8c-7603a073da5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:47.124526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50cda0-73f1-4eb7-8a8c-7603a073da5d admin admin] PATH_INFO: `/v3/users/cec53b13ff744d81ace451af835cf7ad` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:47.127220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50cda0-73f1-4eb7-8a8c-7603a073da5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cec53b13ff744d81ace451af835cf7ad)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:47.129273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50cda0-73f1-4eb7-8a8c-7603a073da5d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:47.185487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-af50cda0-73f1-4eb7-8a8c-7603a073da5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cec53b13ff744d81ace451af835cf7ad'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:47.186943 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-af50cda0-73f1-4eb7-8a8c-7603a073da5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cec53b13ff744d81ace451af835cf7ad'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:47.187707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-af50cda0-73f1-4eb7-8a8c-7603a073da5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cec53b13ff744d81ace451af835cf7ad'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:47.196858 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-af50cda0-73f1-4eb7-8a8c-7603a073da5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cec53b13ff744d81ace451af835cf7ad'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:47.211398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-af50cda0-73f1-4eb7-8a8c-7603a073da5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cec53b13ff744d81ace451af835cf7ad'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:47.236121 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1016/2020] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:47 2022] DELETE /identity/v3/users/cec53b13ff744d81ace451af835cf7ad => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:47.245806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453426e1-ca87-482f-b01c-b0fe96e6c2aa None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:47.246127 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453426e1-ca87-482f-b01c-b0fe96e6c2aa None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:47.246354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453426e1-ca87-482f-b01c-b0fe96e6c2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:47.298655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-453426e1-ca87-482f-b01c-b0fe96e6c2aa None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:47.299972 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-453426e1-ca87-482f-b01c-b0fe96e6c2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:47.412706 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1005/2021] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:47.425693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50cda0-73f1-4eb7-8a8c-7603a073da5d admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:47.427407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-af50cda0-73f1-4eb7-8a8c-7603a073da5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:47.469778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:47.503939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50cda0-73f1-4eb7-8a8c-7603a073da5d admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:47.508687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0ea45-6ca2-496a-b27a-47333e7331f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:47.509527 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0ea45-6ca2-496a-b27a-47333e7331f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:47.509899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0ea45-6ca2-496a-b27a-47333e7331f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:47.512560 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0ea45-6ca2-496a-b27a-47333e7331f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:47.524089 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0ea45-6ca2-496a-b27a-47333e7331f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:47.525660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0ea45-6ca2-496a-b27a-47333e7331f9 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:47.538541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1017/2022] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:47.920728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453426e1-ca87-482f-b01c-b0fe96e6c2aa None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:47.922600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-453426e1-ca87-482f-b01c-b0fe96e6c2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:47.931306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:47.938756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453426e1-ca87-482f-b01c-b0fe96e6c2aa None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:47.949281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:47.950118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:47.950446 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:47.950870 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be admin admin] PATH_INFO: `/v3/projects/50043987841c460d96d1bcb9c7aa8fad` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:47.952280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be admin admin] RBAC: Authorizing `identity:delete_project(project_id=50043987841c460d96d1bcb9c7aa8fad)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:47.954673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:48.015571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50043987841c460d96d1bcb9c7aa8fad', 'request_id': 'req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:48.028213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50043987841c460d96d1bcb9c7aa8fad', 'request_id': 'req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:48.039543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50043987841c460d96d1bcb9c7aa8fad', 'request_id': 'req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:48.054944 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be admin admin] The token cache is being invalidate because project 50043987841c460d96d1bcb9c7aa8fad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:48.055375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:48.056131 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0db0bf69-f6a3-47d8-829f-ee3778f0e7be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:48.058066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1006/2023] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:47 2022] DELETE /identity/v3/projects/50043987841c460d96d1bcb9c7aa8fad => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:48.062239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0ea45-6ca2-496a-b27a-47333e7331f9 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:48.072320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0ea45-6ca2-496a-b27a-47333e7331f9 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:48.167006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2c423-8d19-4b2d-a46c-3a762e064efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:48.181448 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2c423-8d19-4b2d-a46c-3a762e064efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:48.181833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2c423-8d19-4b2d-a46c-3a762e064efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:48.184655 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2c423-8d19-4b2d-a46c-3a762e064efc admin admin] PATH_INFO: `/v3/users/6f299fc102ef4a2f9ed92ea2fc76f93b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:48.189978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2c423-8d19-4b2d-a46c-3a762e064efc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f299fc102ef4a2f9ed92ea2fc76f93b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:48.197282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2c423-8d19-4b2d-a46c-3a762e064efc admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:48.326455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5ef2c423-8d19-4b2d-a46c-3a762e064efc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f299fc102ef4a2f9ed92ea2fc76f93b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:48.328906 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5ef2c423-8d19-4b2d-a46c-3a762e064efc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f299fc102ef4a2f9ed92ea2fc76f93b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:48.329733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5ef2c423-8d19-4b2d-a46c-3a762e064efc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f299fc102ef4a2f9ed92ea2fc76f93b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:48.349491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5ef2c423-8d19-4b2d-a46c-3a762e064efc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f299fc102ef4a2f9ed92ea2fc76f93b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:48.386427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5ef2c423-8d19-4b2d-a46c-3a762e064efc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f299fc102ef4a2f9ed92ea2fc76f93b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:48.406094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1018/2024] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:48 2022] DELETE /identity/v3/users/6f299fc102ef4a2f9ed92ea2fc76f93b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:48.460863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac02c23-894d-4589-a4e2-7e3c3db0c9c4 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:48.461144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac02c23-894d-4589-a4e2-7e3c3db0c9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:48.461464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac02c23-894d-4589-a4e2-7e3c3db0c9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:48.496600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-bac02c23-894d-4589-a4e2-7e3c3db0c9c4 None None] MFA Rules not processed for user `6d2fbc3668cf4a82b8eea0b67b87ca80`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:48.502978 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bac02c23-894d-4589-a4e2-7e3c3db0c9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:48.634873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1007/2025] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:49.007209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2c423-8d19-4b2d-a46c-3a762e064efc admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:49.008676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5ef2c423-8d19-4b2d-a46c-3a762e064efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:49.110115 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:49.132819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2c423-8d19-4b2d-a46c-3a762e064efc admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:49.141821 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:49.142562 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:49.142884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:49.143103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699 admin admin] PATH_INFO: `/v3/projects/5bf63c210c9148d1902752864204e707` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:49.148239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bf63c210c9148d1902752864204e707)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:49.151285 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:49.226452 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bf63c210c9148d1902752864204e707', 'request_id': 'req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:49.242753 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bf63c210c9148d1902752864204e707', 'request_id': 'req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:49.250917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bf63c210c9148d1902752864204e707', 'request_id': 'req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:49.262190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699 admin admin] The token cache is being invalidate because project 5bf63c210c9148d1902752864204e707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:49.262975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:49.263137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e454aa7a-3b3a-49cd-9ab6-df8ce06c1699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:49.264654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1019/2026] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:49 2022] DELETE /identity/v3/projects/5bf63c210c9148d1902752864204e707 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:49.269597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec41327-66ce-4d11-ab23-bfeeaa5cd152 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:49.270078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec41327-66ce-4d11-ab23-bfeeaa5cd152 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:49.270237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec41327-66ce-4d11-ab23-bfeeaa5cd152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:49.296772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-dec41327-66ce-4d11-ab23-bfeeaa5cd152 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:49.298019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dec41327-66ce-4d11-ab23-bfeeaa5cd152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:49.390043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1008/2027] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:49.391843 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bde37-359f-49a0-8a99-54784930fb5d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:49.392130 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bde37-359f-49a0-8a99-54784930fb5d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:49.392401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bde37-359f-49a0-8a99-54784930fb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:49.420873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-5b5bde37-359f-49a0-8a99-54784930fb5d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:49.422175 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5b5bde37-359f-49a0-8a99-54784930fb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:49.431644 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1020/2028] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:49.436264 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec41327-66ce-4d11-ab23-bfeeaa5cd152 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:49.447153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec41327-66ce-4d11-ab23-bfeeaa5cd152 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:49.452384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c060bc8-2f9f-4f85-87f2-dc79e1cda702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:49.453074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c060bc8-2f9f-4f85-87f2-dc79e1cda702 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:49.453542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c060bc8-2f9f-4f85-87f2-dc79e1cda702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:49.453906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c060bc8-2f9f-4f85-87f2-dc79e1cda702 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:49.454776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c060bc8-2f9f-4f85-87f2-dc79e1cda702 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:49.455592 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c060bc8-2f9f-4f85-87f2-dc79e1cda702 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:49.457201 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c060bc8-2f9f-4f85-87f2-dc79e1cda702 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:49.476005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1009/2029] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:32:49.480436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5bde37-359f-49a0-8a99-54784930fb5d None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:49.495423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5bde37-359f-49a0-8a99-54784930fb5d None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:49.500247 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64e65c-2310-4215-9d97-cf9fb217ecd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:49.502818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64e65c-2310-4215-9d97-cf9fb217ecd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:49.502818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64e65c-2310-4215-9d97-cf9fb217ecd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:49.502818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64e65c-2310-4215-9d97-cf9fb217ecd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:49.502818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64e65c-2310-4215-9d97-cf9fb217ecd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:49.504032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64e65c-2310-4215-9d97-cf9fb217ecd9 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:49.563563 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1021/2030] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:49 2022] POST /identity/v3/projects => generated 381 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:49.567965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c060bc8-2f9f-4f85-87f2-dc79e1cda702 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:49.585335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c060bc8-2f9f-4f85-87f2-dc79e1cda702 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:49.603671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:49.670651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd70e1-509f-4e41-9ff2-96819797845e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:49.671278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd70e1-509f-4e41-9ff2-96819797845e admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:49.671560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd70e1-509f-4e41-9ff2-96819797845e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:49.671833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd70e1-509f-4e41-9ff2-96819797845e admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:49.673154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd70e1-509f-4e41-9ff2-96819797845e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:49.675638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd70e1-509f-4e41-9ff2-96819797845e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:49.703743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1010/2031] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:49 2022] POST /identity/v3/users => generated 291 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:49.708140 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64e65c-2310-4215-9d97-cf9fb217ecd9 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:49.718857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64e65c-2310-4215-9d97-cf9fb217ecd9 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:49.723541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746585a-2946-4ac5-9504-115abbe31858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:49.724180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746585a-2946-4ac5-9504-115abbe31858 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:49.724455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746585a-2946-4ac5-9504-115abbe31858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:49.724726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746585a-2946-4ac5-9504-115abbe31858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:49.725404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7746585a-2946-4ac5-9504-115abbe31858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:49.727042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7746585a-2946-4ac5-9504-115abbe31858 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:49.737881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1022/2032] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:32:49.743872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd70e1-509f-4e41-9ff2-96819797845e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:49.754410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd70e1-509f-4e41-9ff2-96819797845e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:49.759530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc809d0-2b5c-43ec-a82d-787e4803f51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:49.760144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc809d0-2b5c-43ec-a82d-787e4803f51a admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:49.760426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc809d0-2b5c-43ec-a82d-787e4803f51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:49.760697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc809d0-2b5c-43ec-a82d-787e4803f51a admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:49.761314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc809d0-2b5c-43ec-a82d-787e4803f51a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:49.762968 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc809d0-2b5c-43ec-a82d-787e4803f51a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:49.774658 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1011/2033] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:49.779130 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746585a-2946-4ac5-9504-115abbe31858 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:49.789926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746585a-2946-4ac5-9504-115abbe31858 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:49.796711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02892223-2407-4c8c-bf8d-9d3dc719f2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:49.797580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02892223-2407-4c8c-bf8d-9d3dc719f2a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:49.797865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02892223-2407-4c8c-bf8d-9d3dc719f2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:49.798138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02892223-2407-4c8c-bf8d-9d3dc719f2a4 admin admin] PATH_INFO: `/v3/projects/0034e8cba58b48c5b4044d156311158f/users/4478d23b1a064e3dbe9b643c04ba0cf1/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:49.814150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02892223-2407-4c8c-bf8d-9d3dc719f2a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0034e8cba58b48c5b4044d156311158f, user_id=4478d23b1a064e3dbe9b643c04ba0cf1, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:49.816857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02892223-2407-4c8c-bf8d-9d3dc719f2a4 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:49.836874 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1023/2034] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:49 2022] PUT /identity/v3/projects/0034e8cba58b48c5b4044d156311158f/users/4478d23b1a064e3dbe9b643c04ba0cf1/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:49.848663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ab0ed-fc94-457d-88e8-7089417cbdc2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:49.848948 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ab0ed-fc94-457d-88e8-7089417cbdc2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:49.849223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ab0ed-fc94-457d-88e8-7089417cbdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:49.883288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-791ab0ed-fc94-457d-88e8-7089417cbdc2 None None] MFA Rules not processed for user `4478d23b1a064e3dbe9b643c04ba0cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:49.884511 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-791ab0ed-fc94-457d-88e8-7089417cbdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:49.982547 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1012/2035] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:49.986192 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02892223-2407-4c8c-bf8d-9d3dc719f2a4 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:50.124705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02892223-2407-4c8c-bf8d-9d3dc719f2a4 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:50.134966 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8420d32-35d4-4877-8295-5e5b0f63d991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:50.135732 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8420d32-35d4-4877-8295-5e5b0f63d991 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:50.136025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8420d32-35d4-4877-8295-5e5b0f63d991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:50.136357 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8420d32-35d4-4877-8295-5e5b0f63d991 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:50.137006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8420d32-35d4-4877-8295-5e5b0f63d991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:50.138953 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8420d32-35d4-4877-8295-5e5b0f63d991 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:50.175326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:50.184492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1024/2036] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:49 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:50.188623 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ab0ed-fc94-457d-88e8-7089417cbdc2 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:50.201294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ab0ed-fc94-457d-88e8-7089417cbdc2 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:50.272994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a76e94-058d-4e7d-91a6-994ecf0faab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:50.273733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a76e94-058d-4e7d-91a6-994ecf0faab0 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:50.273954 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a76e94-058d-4e7d-91a6-994ecf0faab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:50.274202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a76e94-058d-4e7d-91a6-994ecf0faab0 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:50.275083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a76e94-058d-4e7d-91a6-994ecf0faab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:50.277594 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a76e94-058d-4e7d-91a6-994ecf0faab0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:50.313766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1013/2037] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:50 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:32:50.318339 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8420d32-35d4-4877-8295-5e5b0f63d991 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:50.331324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8420d32-35d4-4877-8295-5e5b0f63d991 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:50.336811 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8a331-4495-4f85-8a82-8b4ab39ef62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:50.337455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8a331-4495-4f85-8a82-8b4ab39ef62b admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:50.337711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8a331-4495-4f85-8a82-8b4ab39ef62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:50.337987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8a331-4495-4f85-8a82-8b4ab39ef62b admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:50.338730 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8a331-4495-4f85-8a82-8b4ab39ef62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:50.340655 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8a331-4495-4f85-8a82-8b4ab39ef62b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:50.352878 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1025/2038] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:50.357211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a76e94-058d-4e7d-91a6-994ecf0faab0 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:50.378274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a76e94-058d-4e7d-91a6-994ecf0faab0 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:50.397443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95e639-0f1f-45b2-a1bb-3425352d07c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:50.398429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95e639-0f1f-45b2-a1bb-3425352d07c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:50.398910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95e639-0f1f-45b2-a1bb-3425352d07c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:50.399338 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95e639-0f1f-45b2-a1bb-3425352d07c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:50.400385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95e639-0f1f-45b2-a1bb-3425352d07c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:50.403283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95e639-0f1f-45b2-a1bb-3425352d07c8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:50.415756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1014/2039] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:50.420645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8a331-4495-4f85-8a82-8b4ab39ef62b admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:50.434821 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8a331-4495-4f85-8a82-8b4ab39ef62b admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:50.441319 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca52ed-1257-4e76-a1e5-2cdfaa1d7e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:50.442239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca52ed-1257-4e76-a1e5-2cdfaa1d7e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:50.442550 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca52ed-1257-4e76-a1e5-2cdfaa1d7e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:50.442893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca52ed-1257-4e76-a1e5-2cdfaa1d7e6d admin admin] PATH_INFO: `/v3/projects/c92e565d967b4c19bd256831c5981a5a/users/9cebd7b228aa479caf6f5f9ae41a5ece/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:50.469210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca52ed-1257-4e76-a1e5-2cdfaa1d7e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c92e565d967b4c19bd256831c5981a5a, user_id=9cebd7b228aa479caf6f5f9ae41a5ece, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:50.472179 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca52ed-1257-4e76-a1e5-2cdfaa1d7e6d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:50.491037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1026/2040] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:50 2022] PUT /identity/v3/projects/c92e565d967b4c19bd256831c5981a5a/users/9cebd7b228aa479caf6f5f9ae41a5ece/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:50.501514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a201d-5226-4cf1-b557-e0750d20a223 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:50.501930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a201d-5226-4cf1-b557-e0750d20a223 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:50.502289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a201d-5226-4cf1-b557-e0750d20a223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:50.513606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:50.546781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e88a201d-5226-4cf1-b557-e0750d20a223 None None] MFA Rules not processed for user `9cebd7b228aa479caf6f5f9ae41a5ece`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:50.548950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e88a201d-5226-4cf1-b557-e0750d20a223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:50.625170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1015/2041] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:50.630196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca52ed-1257-4e76-a1e5-2cdfaa1d7e6d admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:50.696458 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca52ed-1257-4e76-a1e5-2cdfaa1d7e6d admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:50.701717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520d752-b870-4558-803b-95be88eac709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:50.702330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520d752-b870-4558-803b-95be88eac709 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:50.702646 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520d752-b870-4558-803b-95be88eac709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:50.702948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520d752-b870-4558-803b-95be88eac709 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:50.703594 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520d752-b870-4558-803b-95be88eac709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:50.705508 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520d752-b870-4558-803b-95be88eac709 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:50.733202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1027/2042] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:50 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:50.737499 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a201d-5226-4cf1-b557-e0750d20a223 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:50.748361 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a201d-5226-4cf1-b557-e0750d20a223 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:50.809292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86e3c8-8a1c-470e-a1ec-cb19cc224489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:50.809894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86e3c8-8a1c-470e-a1ec-cb19cc224489 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:50.810166 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86e3c8-8a1c-470e-a1ec-cb19cc224489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:50.810433 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86e3c8-8a1c-470e-a1ec-cb19cc224489 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:50.811086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86e3c8-8a1c-470e-a1ec-cb19cc224489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:50.812996 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86e3c8-8a1c-470e-a1ec-cb19cc224489 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:50.839788 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1016/2043] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:50 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:32:50.841767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520d752-b870-4558-803b-95be88eac709 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:50.852756 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520d752-b870-4558-803b-95be88eac709 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:50.857761 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23f670-9df3-4168-b8d8-70974e412314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:50.858357 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23f670-9df3-4168-b8d8-70974e412314 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:50.858793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23f670-9df3-4168-b8d8-70974e412314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:50.859373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23f670-9df3-4168-b8d8-70974e412314 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:50.860012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23f670-9df3-4168-b8d8-70974e412314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:50.861604 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23f670-9df3-4168-b8d8-70974e412314 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:50.872241 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1028/2044] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:50.876548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86e3c8-8a1c-470e-a1ec-cb19cc224489 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:50.887888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86e3c8-8a1c-470e-a1ec-cb19cc224489 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:50.894252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb7602-341c-45ba-8a69-f907be95f49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:50.895167 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb7602-341c-45ba-8a69-f907be95f49f admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:50.895501 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb7602-341c-45ba-8a69-f907be95f49f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:50.895833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb7602-341c-45ba-8a69-f907be95f49f admin admin] PATH_INFO: `/v3/projects/759243f2025b4e299382e01ac4d4626d/users/9550be4e80524424b938b665dbcf3318/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:50.917856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb7602-341c-45ba-8a69-f907be95f49f admin admin] RBAC: Authorizing `identity:create_grant(project_id=759243f2025b4e299382e01ac4d4626d, user_id=9550be4e80524424b938b665dbcf3318, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:50.920268 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb7602-341c-45ba-8a69-f907be95f49f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:50.935834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1017/2045] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:50 2022] PUT /identity/v3/projects/759243f2025b4e299382e01ac4d4626d/users/9550be4e80524424b938b665dbcf3318/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:50.940156 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23f670-9df3-4168-b8d8-70974e412314 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:50.952296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23f670-9df3-4168-b8d8-70974e412314 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:51.027943 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcba9b9-77ac-405f-b197-43deef622279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:51.028761 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcba9b9-77ac-405f-b197-43deef622279 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:51.029049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcba9b9-77ac-405f-b197-43deef622279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:51.029326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcba9b9-77ac-405f-b197-43deef622279 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:51.029987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcba9b9-77ac-405f-b197-43deef622279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:51.031728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcba9b9-77ac-405f-b197-43deef622279 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:51.050440 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1029/2046] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:51.051836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb7602-341c-45ba-8a69-f907be95f49f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:51.065162 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb7602-341c-45ba-8a69-f907be95f49f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:51.074868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b124bf-8b75-47fa-8082-c645926356fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:51.075615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b124bf-8b75-47fa-8082-c645926356fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:51.075895 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b124bf-8b75-47fa-8082-c645926356fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:51.076169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b124bf-8b75-47fa-8082-c645926356fe admin admin] PATH_INFO: `/v3/projects/759243f2025b4e299382e01ac4d4626d/users/9550be4e80524424b938b665dbcf3318/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:51.078442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b124bf-8b75-47fa-8082-c645926356fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=759243f2025b4e299382e01ac4d4626d, user_id=9550be4e80524424b938b665dbcf3318, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:51.081177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b124bf-8b75-47fa-8082-c645926356fe admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:51.087854 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcba9b9-77ac-405f-b197-43deef622279 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:51.089689 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ddcba9b9-77ac-405f-b197-43deef622279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:51.107673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcba9b9-77ac-405f-b197-43deef622279 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:51.112490 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1018/2047] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:51 2022] PUT /identity/v3/projects/759243f2025b4e299382e01ac4d4626d/users/9550be4e80524424b938b665dbcf3318/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:51.124975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea97c9-94b3-4a30-8e39-c3bc944582b1 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:51.125180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea97c9-94b3-4a30-8e39-c3bc944582b1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:51.125397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea97c9-94b3-4a30-8e39-c3bc944582b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:51.163204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:51.175261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3fea97c9-94b3-4a30-8e39-c3bc944582b1 None None] MFA Rules not processed for user `9550be4e80524424b938b665dbcf3318`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:51.176532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3fea97c9-94b3-4a30-8e39-c3bc944582b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:51.195777 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99747f6d-a8d0-47b8-b45e-e18d4e9e0b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:51.196617 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99747f6d-a8d0-47b8-b45e-e18d4e9e0b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:51.196891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99747f6d-a8d0-47b8-b45e-e18d4e9e0b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:51.197180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99747f6d-a8d0-47b8-b45e-e18d4e9e0b80 admin admin] PATH_INFO: `/v3/users/2ddf4cfa095440f6a5b46c382556843f` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:51.198814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99747f6d-a8d0-47b8-b45e-e18d4e9e0b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ddf4cfa095440f6a5b46c382556843f)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:51.201164 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99747f6d-a8d0-47b8-b45e-e18d4e9e0b80 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:51.222160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:51.260219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-99747f6d-a8d0-47b8-b45e-e18d4e9e0b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ddf4cfa095440f6a5b46c382556843f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:51.261025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-99747f6d-a8d0-47b8-b45e-e18d4e9e0b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ddf4cfa095440f6a5b46c382556843f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:51.261733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-99747f6d-a8d0-47b8-b45e-e18d4e9e0b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ddf4cfa095440f6a5b46c382556843f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:51.273413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-99747f6d-a8d0-47b8-b45e-e18d4e9e0b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ddf4cfa095440f6a5b46c382556843f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:51.277832 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1019/2048] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:51.289274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea97c9-94b3-4a30-8e39-c3bc944582b1 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:51.290733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3fea97c9-94b3-4a30-8e39-c3bc944582b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:51.292344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-99747f6d-a8d0-47b8-b45e-e18d4e9e0b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ddf4cfa095440f6a5b46c382556843f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:51.315116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1030/2049] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:51 2022] DELETE /identity/v3/users/2ddf4cfa095440f6a5b46c382556843f => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:51.319934 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea31321-82df-4ba5-9a2e-62ddb7032964 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:51.320358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea31321-82df-4ba5-9a2e-62ddb7032964 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:51.320664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea31321-82df-4ba5-9a2e-62ddb7032964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:51.346944 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-0ea31321-82df-4ba5-9a2e-62ddb7032964 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:51.348596 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0ea31321-82df-4ba5-9a2e-62ddb7032964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:51.377272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea97c9-94b3-4a30-8e39-c3bc944582b1 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:51.436160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3d7ae-7ae6-4f16-8d52-c0ab2751200c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:51.436838 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3d7ae-7ae6-4f16-8d52-c0ab2751200c service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:51.437059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3d7ae-7ae6-4f16-8d52-c0ab2751200c service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:51.437329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3d7ae-7ae6-4f16-8d52-c0ab2751200c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:51.439139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1031/2050] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:51.439345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ace3d7ae-7ae6-4f16-8d52-c0ab2751200c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:51.453808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea31321-82df-4ba5-9a2e-62ddb7032964 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:51.456293 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0ea31321-82df-4ba5-9a2e-62ddb7032964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:51.514647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3d7ae-7ae6-4f16-8d52-c0ab2751200c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:51.516117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3d7ae-7ae6-4f16-8d52-c0ab2751200c service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:51.528638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1020/2051] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:51.530166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea31321-82df-4ba5-9a2e-62ddb7032964 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:51.533875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f56e8-32b4-4246-8e11-b5bb46ff0816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:51.534585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f56e8-32b4-4246-8e11-b5bb46ff0816 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:51.534924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f56e8-32b4-4246-8e11-b5bb46ff0816 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:51.535255 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f56e8-32b4-4246-8e11-b5bb46ff0816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:51.544893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f56e8-32b4-4246-8e11-b5bb46ff0816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:51.546423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f56e8-32b4-4246-8e11-b5bb46ff0816 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:51.551034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:51.559519 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1032/2052] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:52.127902 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3d7ae-7ae6-4f16-8d52-c0ab2751200c service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:52.129617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ace3d7ae-7ae6-4f16-8d52-c0ab2751200c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:52.151854 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3d7ae-7ae6-4f16-8d52-c0ab2751200c service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:52.179373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551af28-db25-46ef-a098-32edcb851afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:52.181059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551af28-db25-46ef-a098-32edcb851afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:52.181443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551af28-db25-46ef-a098-32edcb851afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:52.181785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551af28-db25-46ef-a098-32edcb851afc admin admin] PATH_INFO: `/v3/projects/abb443b8dc364b16ad535e840b5bc8cd` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:52.183511 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551af28-db25-46ef-a098-32edcb851afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb443b8dc364b16ad535e840b5bc8cd)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:52.185671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551af28-db25-46ef-a098-32edcb851afc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:52.234321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:52.261128 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a551af28-db25-46ef-a098-32edcb851afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb443b8dc364b16ad535e840b5bc8cd', 'request_id': 'req-a551af28-db25-46ef-a098-32edcb851afc'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:52.269473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a551af28-db25-46ef-a098-32edcb851afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb443b8dc364b16ad535e840b5bc8cd', 'request_id': 'req-a551af28-db25-46ef-a098-32edcb851afc'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:52.272625 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f56e8-32b4-4246-8e11-b5bb46ff0816 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:52.274100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ca5f56e8-32b4-4246-8e11-b5bb46ff0816 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:52.276495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a551af28-db25-46ef-a098-32edcb851afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb443b8dc364b16ad535e840b5bc8cd', 'request_id': 'req-a551af28-db25-46ef-a098-32edcb851afc'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:52.293814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f56e8-32b4-4246-8e11-b5bb46ff0816 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:52.302252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a551af28-db25-46ef-a098-32edcb851afc admin admin] The token cache is being invalidate because project abb443b8dc364b16ad535e840b5bc8cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:52.303375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a551af28-db25-46ef-a098-32edcb851afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:52.307013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a551af28-db25-46ef-a098-32edcb851afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:52.310247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1021/2053] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:52 2022] DELETE /identity/v3/projects/abb443b8dc364b16ad535e840b5bc8cd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:52.316380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb12e2f-e01e-4b9f-8a49-e7f222ce8089 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:52.316669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb12e2f-e01e-4b9f-8a49-e7f222ce8089 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:52.318558 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb12e2f-e01e-4b9f-8a49-e7f222ce8089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:52.383923 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-deb12e2f-e01e-4b9f-8a49-e7f222ce8089 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:52.385682 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-deb12e2f-e01e-4b9f-8a49-e7f222ce8089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:52.435737 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9d8c7-8f74-4c82-8acd-74e6e8649bc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:52.436631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9d8c7-8f74-4c82-8acd-74e6e8649bc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:52.437865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9d8c7-8f74-4c82-8acd-74e6e8649bc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:52.438325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9d8c7-8f74-4c82-8acd-74e6e8649bc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:52.441652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e1e9d8c7-8f74-4c82-8acd-74e6e8649bc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:52.534564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1022/2054] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:52.538763 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc031392-75ad-4430-abc1-2b2706586127 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:52.539160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc031392-75ad-4430-abc1-2b2706586127 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:52.539635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc031392-75ad-4430-abc1-2b2706586127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:52.557567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e9d8c7-8f74-4c82-8acd-74e6e8649bc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:52.560106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e9d8c7-8f74-4c82-8acd-74e6e8649bc8 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:52.594187 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-dc031392-75ad-4430-abc1-2b2706586127 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:52.595952 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dc031392-75ad-4430-abc1-2b2706586127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:52.611315 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1033/2055] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:32:52.617654 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1023/2056] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:52.618982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9d8c7-8f74-4c82-8acd-74e6e8649bc8 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:52.645492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9d8c7-8f74-4c82-8acd-74e6e8649bc8 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:52.662248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce737f3-a742-400b-adc5-3307635bbaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:52.663247 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce737f3-a742-400b-adc5-3307635bbaf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:52.663537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce737f3-a742-400b-adc5-3307635bbaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:52.663819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce737f3-a742-400b-adc5-3307635bbaf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:52.664829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce737f3-a742-400b-adc5-3307635bbaf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:52.665143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce737f3-a742-400b-adc5-3307635bbaf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:52.667626 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce737f3-a742-400b-adc5-3307635bbaf2 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:52.682595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1034/2057] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:32:52.687024 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc031392-75ad-4430-abc1-2b2706586127 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:52.700470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc031392-75ad-4430-abc1-2b2706586127 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:52.707105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697e404-25b0-4dd2-a6fa-da125858246e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:52.708950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697e404-25b0-4dd2-a6fa-da125858246e admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:52.709311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697e404-25b0-4dd2-a6fa-da125858246e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:52.709725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697e404-25b0-4dd2-a6fa-da125858246e admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:52.718709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697e404-25b0-4dd2-a6fa-da125858246e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:52.718709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697e404-25b0-4dd2-a6fa-da125858246e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:52.810764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1024/2058] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:52 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:32:52.815349 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce737f3-a742-400b-adc5-3307635bbaf2 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:52.831216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce737f3-a742-400b-adc5-3307635bbaf2 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:52.943193 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e2538-6b61-42e6-aa39-f36924ab32a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:52.943988 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2538-6b61-42e6-aa39-f36924ab32a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:52.944269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2538-6b61-42e6-aa39-f36924ab32a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:52.944582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2538-6b61-42e6-aa39-f36924ab32a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:52.945242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e2538-6b61-42e6-aa39-f36924ab32a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:52.947384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e2538-6b61-42e6-aa39-f36924ab32a4 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:52.979150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1035/2059] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:52 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:52.980434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697e404-25b0-4dd2-a6fa-da125858246e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:52.991528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697e404-25b0-4dd2-a6fa-da125858246e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:52.997439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52af2c-9705-4f79-bbc0-113838c99fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:52.998040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52af2c-9705-4f79-bbc0-113838c99fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:52.998313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52af2c-9705-4f79-bbc0-113838c99fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:52.998613 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52af2c-9705-4f79-bbc0-113838c99fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:52.999251 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52af2c-9705-4f79-bbc0-113838c99fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:53.001247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52af2c-9705-4f79-bbc0-113838c99fc5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:53.011765 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1025/2060] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:32:53.016141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e2538-6b61-42e6-aa39-f36924ab32a4 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:53.039871 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e2538-6b61-42e6-aa39-f36924ab32a4 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:53.046460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e14ab4-763e-410e-8a98-4e9b2758f82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:53.047348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e14ab4-763e-410e-8a98-4e9b2758f82d admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:53.047714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e14ab4-763e-410e-8a98-4e9b2758f82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:53.048051 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e14ab4-763e-410e-8a98-4e9b2758f82d admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:53.049239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e14ab4-763e-410e-8a98-4e9b2758f82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:53.052419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e14ab4-763e-410e-8a98-4e9b2758f82d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:53.090171 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52af2c-9705-4f79-bbc0-113838c99fc5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:53.106796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52af2c-9705-4f79-bbc0-113838c99fc5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:53.107104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1036/2061] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:53.119997 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d3341-05bd-42ee-b09a-c4c876469339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:53.120345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d3341-05bd-42ee-b09a-c4c876469339 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:53.120750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d3341-05bd-42ee-b09a-c4c876469339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:53.120853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d3341-05bd-42ee-b09a-c4c876469339 admin admin] PATH_INFO: `/v3/projects/47baef5431ac4ccf8240a3ad791fe391/users/b40c608b7b504fe897cd79007cffb3f2/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:53.158749 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d3341-05bd-42ee-b09a-c4c876469339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47baef5431ac4ccf8240a3ad791fe391, user_id=b40c608b7b504fe897cd79007cffb3f2, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:53.161826 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d3341-05bd-42ee-b09a-c4c876469339 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:53.186468 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1026/2062] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:53 2022] PUT /identity/v3/projects/47baef5431ac4ccf8240a3ad791fe391/users/b40c608b7b504fe897cd79007cffb3f2/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:32:53.198387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b68b65-9eb9-4d73-b5f1-c4ac8fc0abcc None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:53.199104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b68b65-9eb9-4d73-b5f1-c4ac8fc0abcc None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:53.199576 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b68b65-9eb9-4d73-b5f1-c4ac8fc0abcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:53.240530 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-01b68b65-9eb9-4d73-b5f1-c4ac8fc0abcc None None] MFA Rules not processed for user `b40c608b7b504fe897cd79007cffb3f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:53.242893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-01b68b65-9eb9-4d73-b5f1-c4ac8fc0abcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:53.250783 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:53.360105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1037/2063] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:53.368716 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d3341-05bd-42ee-b09a-c4c876469339 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:53.370718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f44d3341-05bd-42ee-b09a-c4c876469339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:53.441884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d3341-05bd-42ee-b09a-c4c876469339 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:53.446392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fd786-9fd5-4087-b9ba-11c24691f107 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:53.447282 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fd786-9fd5-4087-b9ba-11c24691f107 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:53.447512 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fd786-9fd5-4087-b9ba-11c24691f107 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:53.447836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fd786-9fd5-4087-b9ba-11c24691f107 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:53.457481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8fd786-9fd5-4087-b9ba-11c24691f107 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:53.459165 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8fd786-9fd5-4087-b9ba-11c24691f107 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:53.464170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:53.469476 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1027/2064] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:56.258005 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b68b65-9eb9-4d73-b5f1-c4ac8fc0abcc None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:56.339652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b68b65-9eb9-4d73-b5f1-c4ac8fc0abcc None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:56.344942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c0924-447f-4463-8fdc-5cb99efc321d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:56.345714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c0924-447f-4463-8fdc-5cb99efc321d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:56.345995 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c0924-447f-4463-8fdc-5cb99efc321d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:56.346268 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c0924-447f-4463-8fdc-5cb99efc321d admin admin] PATH_INFO: `/v3/users/b40c608b7b504fe897cd79007cffb3f2` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:56.347509 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c0924-447f-4463-8fdc-5cb99efc321d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b40c608b7b504fe897cd79007cffb3f2)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:56.349692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c0924-447f-4463-8fdc-5cb99efc321d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:56.422493 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-199c0924-447f-4463-8fdc-5cb99efc321d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b40c608b7b504fe897cd79007cffb3f2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:56.423902 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-199c0924-447f-4463-8fdc-5cb99efc321d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b40c608b7b504fe897cd79007cffb3f2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:56.424687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-199c0924-447f-4463-8fdc-5cb99efc321d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b40c608b7b504fe897cd79007cffb3f2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:56.434289 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-199c0924-447f-4463-8fdc-5cb99efc321d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b40c608b7b504fe897cd79007cffb3f2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:56.450891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-199c0924-447f-4463-8fdc-5cb99efc321d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b40c608b7b504fe897cd79007cffb3f2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:56.475364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1038/2065] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:56 2022] DELETE /identity/v3/users/b40c608b7b504fe897cd79007cffb3f2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:56.479924 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4e200-8fb1-482a-ad63-37734f90f2c9 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:56.480438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4e200-8fb1-482a-ad63-37734f90f2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:56.480438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4e200-8fb1-482a-ad63-37734f90f2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:56.500325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-31a4e200-8fb1-482a-ad63-37734f90f2c9 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:56.501537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-31a4e200-8fb1-482a-ad63-37734f90f2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:56.585834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1028/2066] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:56.601762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c0924-447f-4463-8fdc-5cb99efc321d admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:56.603362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-199c0924-447f-4463-8fdc-5cb99efc321d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:56.659866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:56.687504 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c0924-447f-4463-8fdc-5cb99efc321d admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:56.691097 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebbe61-7aba-41ac-b41f-3726e974289f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:56.691748 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebbe61-7aba-41ac-b41f-3726e974289f service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:56.692081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebbe61-7aba-41ac-b41f-3726e974289f service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:56.692352 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebbe61-7aba-41ac-b41f-3726e974289f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:56.702067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebbe61-7aba-41ac-b41f-3726e974289f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:56.703627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebbe61-7aba-41ac-b41f-3726e974289f service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:56.714017 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1039/2067] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:57.009881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4e200-8fb1-482a-ad63-37734f90f2c9 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:57.011677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-31a4e200-8fb1-482a-ad63-37734f90f2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:57.036705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4e200-8fb1-482a-ad63-37734f90f2c9 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:57.044284 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a8156-bac6-4c03-bc07-29b555f9e60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:57.045143 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a8156-bac6-4c03-bc07-29b555f9e60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:57.045814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a8156-bac6-4c03-bc07-29b555f9e60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:57.046354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a8156-bac6-4c03-bc07-29b555f9e60c admin admin] PATH_INFO: `/v3/projects/47baef5431ac4ccf8240a3ad791fe391` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:57.047562 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a8156-bac6-4c03-bc07-29b555f9e60c admin admin] RBAC: Authorizing `identity:delete_project(project_id=47baef5431ac4ccf8240a3ad791fe391)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:57.049856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a8156-bac6-4c03-bc07-29b555f9e60c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:57.115490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebbe61-7aba-41ac-b41f-3726e974289f service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:57.116899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-6cebbe61-7aba-41ac-b41f-3726e974289f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:57.117920 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d79a8156-bac6-4c03-bc07-29b555f9e60c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47baef5431ac4ccf8240a3ad791fe391', 'request_id': 'req-d79a8156-bac6-4c03-bc07-29b555f9e60c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:57.127330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d79a8156-bac6-4c03-bc07-29b555f9e60c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47baef5431ac4ccf8240a3ad791fe391', 'request_id': 'req-d79a8156-bac6-4c03-bc07-29b555f9e60c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:57.139991 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d79a8156-bac6-4c03-bc07-29b555f9e60c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47baef5431ac4ccf8240a3ad791fe391', 'request_id': 'req-d79a8156-bac6-4c03-bc07-29b555f9e60c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:57.159958 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d79a8156-bac6-4c03-bc07-29b555f9e60c admin admin] The token cache is being invalidate because project 47baef5431ac4ccf8240a3ad791fe391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:32:57.160245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d79a8156-bac6-4c03-bc07-29b555f9e60c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:57.161283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d79a8156-bac6-4c03-bc07-29b555f9e60c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:57.163197 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1029/2068] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:32:57 2022] DELETE /identity/v3/projects/47baef5431ac4ccf8240a3ad791fe391 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:57.168711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a28f2-76d1-4b83-b01d-d03108bba771 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:57.169187 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a28f2-76d1-4b83-b01d-d03108bba771 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:57.169568 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a28f2-76d1-4b83-b01d-d03108bba771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:57.180251 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebbe61-7aba-41ac-b41f-3726e974289f service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:57.192184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ee4a28f2-76d1-4b83-b01d-d03108bba771 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:57.193398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ee4a28f2-76d1-4b83-b01d-d03108bba771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:57.233080 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd5d8c-f1be-4516-b6a6-17e3254a136e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:57.233720 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd5d8c-f1be-4516-b6a6-17e3254a136e service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:57.234031 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd5d8c-f1be-4516-b6a6-17e3254a136e service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:57.234301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd5d8c-f1be-4516-b6a6-17e3254a136e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:57.236127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-03dd5d8c-f1be-4516-b6a6-17e3254a136e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:57.274651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1030/2069] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:32:57.279376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96e99c-821c-4427-86de-02a212976ca9 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:57.279747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96e99c-821c-4427-86de-02a212976ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:57.280120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96e99c-821c-4427-86de-02a212976ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:57.299367 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd5d8c-f1be-4516-b6a6-17e3254a136e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:57.300820 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd5d8c-f1be-4516-b6a6-17e3254a136e service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:57.312214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1040/2070] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:57.317101 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-2d96e99c-821c-4427-86de-02a212976ca9 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:57.318580 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2d96e99c-821c-4427-86de-02a212976ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:57.327894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1031/2071] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:57.332810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd5d8c-f1be-4516-b6a6-17e3254a136e service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:57.344821 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd5d8c-f1be-4516-b6a6-17e3254a136e service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:57.349408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7116ff-791b-4a9a-b065-be5e3e267268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:57.350010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7116ff-791b-4a9a-b065-be5e3e267268 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:57.350284 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7116ff-791b-4a9a-b065-be5e3e267268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:57.350588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7116ff-791b-4a9a-b065-be5e3e267268 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:57.351311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7116ff-791b-4a9a-b065-be5e3e267268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:32:57.351609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7116ff-791b-4a9a-b065-be5e3e267268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:57.353195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7116ff-791b-4a9a-b065-be5e3e267268 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:57.365200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1041/2072] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:32:57.369554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96e99c-821c-4427-86de-02a212976ca9 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:57.378728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96e99c-821c-4427-86de-02a212976ca9 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:57.383424 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b2158-1d73-430e-a4f5-2213b56439c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:57.384021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b2158-1d73-430e-a4f5-2213b56439c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:57.384295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b2158-1d73-430e-a4f5-2213b56439c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:57.384559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b2158-1d73-430e-a4f5-2213b56439c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:57.385174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b2158-1d73-430e-a4f5-2213b56439c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:57.387104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b2158-1d73-430e-a4f5-2213b56439c2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:57.402781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:57.420818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1032/2073] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:32:57 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:57.422066 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7116ff-791b-4a9a-b065-be5e3e267268 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:57.433917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7116ff-791b-4a9a-b065-be5e3e267268 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:57.462588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:57.497424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16be2ad-93e0-470b-9c7f-d6bb2140edbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:57.498041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16be2ad-93e0-470b-9c7f-d6bb2140edbf admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:57.498320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16be2ad-93e0-470b-9c7f-d6bb2140edbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:57.498626 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16be2ad-93e0-470b-9c7f-d6bb2140edbf admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:57.499268 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16be2ad-93e0-470b-9c7f-d6bb2140edbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:57.501202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16be2ad-93e0-470b-9c7f-d6bb2140edbf admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:57.524476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1042/2074] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:32:57 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:32:57.528847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b2158-1d73-430e-a4f5-2213b56439c2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:57.540022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b2158-1d73-430e-a4f5-2213b56439c2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:57.544912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13737dd9-554b-4aa8-a337-fcad18570da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:57.545510 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13737dd9-554b-4aa8-a337-fcad18570da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:57.545783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13737dd9-554b-4aa8-a337-fcad18570da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:57.546048 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13737dd9-554b-4aa8-a337-fcad18570da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:57.546818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13737dd9-554b-4aa8-a337-fcad18570da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:57.548715 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13737dd9-554b-4aa8-a337-fcad18570da2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:57.559613 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1033/2075] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:57.563877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16be2ad-93e0-470b-9c7f-d6bb2140edbf admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:57.574614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16be2ad-93e0-470b-9c7f-d6bb2140edbf admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:57.579700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7cc6b-2efa-46a3-a75d-429bcb0b1914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:57.580304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7cc6b-2efa-46a3-a75d-429bcb0b1914 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:57.580612 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7cc6b-2efa-46a3-a75d-429bcb0b1914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:57.580882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7cc6b-2efa-46a3-a75d-429bcb0b1914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:57.581531 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7cc6b-2efa-46a3-a75d-429bcb0b1914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:57.583306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7cc6b-2efa-46a3-a75d-429bcb0b1914 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:57.594463 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1043/2076] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:32:57.598960 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13737dd9-554b-4aa8-a337-fcad18570da2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:57.608998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13737dd9-554b-4aa8-a337-fcad18570da2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:57.613658 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555cc98-a8a7-4c43-a55d-95c44c8fc3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:57.614316 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555cc98-a8a7-4c43-a55d-95c44c8fc3fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:57.615277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555cc98-a8a7-4c43-a55d-95c44c8fc3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:57.616597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555cc98-a8a7-4c43-a55d-95c44c8fc3fc admin admin] PATH_INFO: `/v3/projects/d862d97a5aaa4ea3976f7d7768cd45b4/users/2f9f213cb055481cbd5b018acbd0cde5/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:57.633829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555cc98-a8a7-4c43-a55d-95c44c8fc3fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d862d97a5aaa4ea3976f7d7768cd45b4, user_id=2f9f213cb055481cbd5b018acbd0cde5, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:57.637274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555cc98-a8a7-4c43-a55d-95c44c8fc3fc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:57.652801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1034/2077] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:32:57 2022] PUT /identity/v3/projects/d862d97a5aaa4ea3976f7d7768cd45b4/users/2f9f213cb055481cbd5b018acbd0cde5/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:57.663626 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8a8a4-a2a6-473d-bfe6-741ab213871a None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:57.663933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8a8a4-a2a6-473d-bfe6-741ab213871a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:57.664199 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8a8a4-a2a6-473d-bfe6-741ab213871a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:57.697750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-c9c8a8a4-a2a6-473d-bfe6-741ab213871a None None] MFA Rules not processed for user `2f9f213cb055481cbd5b018acbd0cde5`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:57.700074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c9c8a8a4-a2a6-473d-bfe6-741ab213871a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:57.771241 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1044/2078] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:57.782616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555cc98-a8a7-4c43-a55d-95c44c8fc3fc admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:57.784523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f555cc98-a8a7-4c43-a55d-95c44c8fc3fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:57.862021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555cc98-a8a7-4c43-a55d-95c44c8fc3fc admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:57.865887 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f74169-7ec6-44c7-af13-1db627ab5998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:57.866530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f74169-7ec6-44c7-af13-1db627ab5998 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:57.866805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f74169-7ec6-44c7-af13-1db627ab5998 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:57.867080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f74169-7ec6-44c7-af13-1db627ab5998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:57.876569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f74169-7ec6-44c7-af13-1db627ab5998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:57.878036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f74169-7ec6-44c7-af13-1db627ab5998 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:57.887500 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1035/2079] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:32:59.682936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8a8a4-a2a6-473d-bfe6-741ab213871a None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:59.684418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c9c8a8a4-a2a6-473d-bfe6-741ab213871a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:59.777801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8a8a4-a2a6-473d-bfe6-741ab213871a None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:32:59.783375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf117f-d05e-4f2b-85bc-8d02784563e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:32:59.783671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf117f-d05e-4f2b-85bc-8d02784563e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:59.783987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf117f-d05e-4f2b-85bc-8d02784563e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:59.784302 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf117f-d05e-4f2b-85bc-8d02784563e1 admin admin] PATH_INFO: `/v3/users/4478d23b1a064e3dbe9b643c04ba0cf1` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:59.785511 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf117f-d05e-4f2b-85bc-8d02784563e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4478d23b1a064e3dbe9b643c04ba0cf1)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:32:59.787842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf117f-d05e-4f2b-85bc-8d02784563e1 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:32:59.829618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:32:59.840195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-92cf117f-d05e-4f2b-85bc-8d02784563e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4478d23b1a064e3dbe9b643c04ba0cf1'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:59.840869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-92cf117f-d05e-4f2b-85bc-8d02784563e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4478d23b1a064e3dbe9b643c04ba0cf1'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:59.841642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-92cf117f-d05e-4f2b-85bc-8d02784563e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4478d23b1a064e3dbe9b643c04ba0cf1'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:59.853939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-92cf117f-d05e-4f2b-85bc-8d02784563e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4478d23b1a064e3dbe9b643c04ba0cf1'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:59.867223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-92cf117f-d05e-4f2b-85bc-8d02784563e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4478d23b1a064e3dbe9b643c04ba0cf1'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:32:59.886562 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1045/2080] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:32:59 2022] DELETE /identity/v3/users/4478d23b1a064e3dbe9b643c04ba0cf1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:32:59.888278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9dd7ec-fbd1-462d-b1eb-5c7f982885e1 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:32:59.888824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9dd7ec-fbd1-462d-b1eb-5c7f982885e1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:32:59.888824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9dd7ec-fbd1-462d-b1eb-5c7f982885e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:32:59.908127 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3f9dd7ec-fbd1-462d-b1eb-5c7f982885e1 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:32:59.909335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3f9dd7ec-fbd1-462d-b1eb-5c7f982885e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:32:59.969750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1036/2081] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:32:59.979908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf117f-d05e-4f2b-85bc-8d02784563e1 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:32:59.981577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-92cf117f-d05e-4f2b-85bc-8d02784563e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:00.042627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf117f-d05e-4f2b-85bc-8d02784563e1 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:00.046931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473f4bb-1baa-44f4-9846-c51f61fdac69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:00.047556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473f4bb-1baa-44f4-9846-c51f61fdac69 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:00.047834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473f4bb-1baa-44f4-9846-c51f61fdac69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:00.048491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473f4bb-1baa-44f4-9846-c51f61fdac69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:00.058945 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473f4bb-1baa-44f4-9846-c51f61fdac69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:00.060446 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473f4bb-1baa-44f4-9846-c51f61fdac69 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:00.070843 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1046/2082] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:33:00.367695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9dd7ec-fbd1-462d-b1eb-5c7f982885e1 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:00.369161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3f9dd7ec-fbd1-462d-b1eb-5c7f982885e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:00.387060 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9dd7ec-fbd1-462d-b1eb-5c7f982885e1 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:00.392508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:00.393152 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:00.393448 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:00.393670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] PATH_INFO: `/v3/projects/0034e8cba58b48c5b4044d156311158f` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:00.395236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=0034e8cba58b48c5b4044d156311158f)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:00.397359 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:00.449998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0034e8cba58b48c5b4044d156311158f', 'request_id': 'req-82a9ef4f-4426-49ef-88da-d56afb0611ce'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:00.456792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0034e8cba58b48c5b4044d156311158f', 'request_id': 'req-82a9ef4f-4426-49ef-88da-d56afb0611ce'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:00.461579 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:00.464045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0034e8cba58b48c5b4044d156311158f', 'request_id': 'req-82a9ef4f-4426-49ef-88da-d56afb0611ce'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:00.480147 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] The token cache is being invalidate because project 0034e8cba58b48c5b4044d156311158f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:33:00.480427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:00.481055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:00.482614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1037/2083] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:33:00 2022] DELETE /identity/v3/projects/0034e8cba58b48c5b4044d156311158f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:00.486767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473f4bb-1baa-44f4-9846-c51f61fdac69 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:00.496525 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473f4bb-1baa-44f4-9846-c51f61fdac69 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:00.561035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66e45a-9c90-4ba4-9e9d-d1a5a3de2939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:00.561686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66e45a-9c90-4ba4-9e9d-d1a5a3de2939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:00.562078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66e45a-9c90-4ba4-9e9d-d1a5a3de2939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:00.562356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66e45a-9c90-4ba4-9e9d-d1a5a3de2939 admin admin] PATH_INFO: `/v3/users/9cebd7b228aa479caf6f5f9ae41a5ece` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:00.563613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf66e45a-9c90-4ba4-9e9d-d1a5a3de2939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cebd7b228aa479caf6f5f9ae41a5ece)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:00.565573 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf66e45a-9c90-4ba4-9e9d-d1a5a3de2939 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:00.613717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bf66e45a-9c90-4ba4-9e9d-d1a5a3de2939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cebd7b228aa479caf6f5f9ae41a5ece'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:00.614269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bf66e45a-9c90-4ba4-9e9d-d1a5a3de2939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cebd7b228aa479caf6f5f9ae41a5ece'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:00.614905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bf66e45a-9c90-4ba4-9e9d-d1a5a3de2939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cebd7b228aa479caf6f5f9ae41a5ece'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:00.623546 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bf66e45a-9c90-4ba4-9e9d-d1a5a3de2939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cebd7b228aa479caf6f5f9ae41a5ece'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:00.625492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:00.638354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bf66e45a-9c90-4ba4-9e9d-d1a5a3de2939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cebd7b228aa479caf6f5f9ae41a5ece'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:00.654193 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1047/2084] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:33:00 2022] DELETE /identity/v3/users/9cebd7b228aa479caf6f5f9ae41a5ece => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:01.014263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:01.015776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:01.030762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9ef4f-4426-49ef-88da-d56afb0611ce admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:01.090040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:01.090723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:01.090999 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:01.091290 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] PATH_INFO: `/v3/projects/c92e565d967b4c19bd256831c5981a5a` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:01.092453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c92e565d967b4c19bd256831c5981a5a)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:01.094388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:01.142715 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c92e565d967b4c19bd256831c5981a5a', 'request_id': 'req-2e68b8be-399d-4400-b3c7-aa10a343b575'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.149053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c92e565d967b4c19bd256831c5981a5a', 'request_id': 'req-2e68b8be-399d-4400-b3c7-aa10a343b575'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.155196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c92e565d967b4c19bd256831c5981a5a', 'request_id': 'req-2e68b8be-399d-4400-b3c7-aa10a343b575'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.168873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] The token cache is being invalidate because project c92e565d967b4c19bd256831c5981a5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:33:01.169150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.169750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.171402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1038/2085] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:33:01 2022] DELETE /identity/v3/projects/c92e565d967b4c19bd256831c5981a5a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:33:01.175782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66e45a-9c90-4ba4-9e9d-d1a5a3de2939 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:01.177622 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-bf66e45a-9c90-4ba4-9e9d-d1a5a3de2939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:01.192306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66e45a-9c90-4ba4-9e9d-d1a5a3de2939 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:01.258325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb298a2-50e3-4772-9fd6-df7729e4da0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:01.259091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb298a2-50e3-4772-9fd6-df7729e4da0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:01.259447 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb298a2-50e3-4772-9fd6-df7729e4da0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:01.259686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb298a2-50e3-4772-9fd6-df7729e4da0a admin admin] PATH_INFO: `/v3/users/9550be4e80524424b938b665dbcf3318` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:01.260845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb298a2-50e3-4772-9fd6-df7729e4da0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9550be4e80524424b938b665dbcf3318)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:01.262810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb298a2-50e3-4772-9fd6-df7729e4da0a admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:01.309892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-deb298a2-50e3-4772-9fd6-df7729e4da0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9550be4e80524424b938b665dbcf3318'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.310568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-deb298a2-50e3-4772-9fd6-df7729e4da0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9550be4e80524424b938b665dbcf3318'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.311166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-deb298a2-50e3-4772-9fd6-df7729e4da0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9550be4e80524424b938b665dbcf3318'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.318072 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-deb298a2-50e3-4772-9fd6-df7729e4da0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9550be4e80524424b938b665dbcf3318'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.329739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-deb298a2-50e3-4772-9fd6-df7729e4da0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9550be4e80524424b938b665dbcf3318'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.345470 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1048/2086] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:33:01 2022] DELETE /identity/v3/users/9550be4e80524424b938b665dbcf3318 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:01.631945 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:01.633549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:01.651032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68b8be-399d-4400-b3c7-aa10a343b575 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:01.709936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9975ae-318c-43f6-a4c3-82300ed56006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:01.710615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9975ae-318c-43f6-a4c3-82300ed56006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:01.711088 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9975ae-318c-43f6-a4c3-82300ed56006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:01.711208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9975ae-318c-43f6-a4c3-82300ed56006 admin admin] PATH_INFO: `/v3/projects/759243f2025b4e299382e01ac4d4626d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:01.712292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9975ae-318c-43f6-a4c3-82300ed56006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=759243f2025b4e299382e01ac4d4626d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:01.714268 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9975ae-318c-43f6-a4c3-82300ed56006 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:01.735516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:01.764633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bc9975ae-318c-43f6-a4c3-82300ed56006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '759243f2025b4e299382e01ac4d4626d', 'request_id': 'req-bc9975ae-318c-43f6-a4c3-82300ed56006'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.771316 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bc9975ae-318c-43f6-a4c3-82300ed56006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '759243f2025b4e299382e01ac4d4626d', 'request_id': 'req-bc9975ae-318c-43f6-a4c3-82300ed56006'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.778839 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bc9975ae-318c-43f6-a4c3-82300ed56006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '759243f2025b4e299382e01ac4d4626d', 'request_id': 'req-bc9975ae-318c-43f6-a4c3-82300ed56006'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.795409 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bc9975ae-318c-43f6-a4c3-82300ed56006 admin admin] The token cache is being invalidate because project 759243f2025b4e299382e01ac4d4626d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:33:01.795747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bc9975ae-318c-43f6-a4c3-82300ed56006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.796650 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bc9975ae-318c-43f6-a4c3-82300ed56006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:01.799432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1039/2087] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:33:01 2022] DELETE /identity/v3/projects/759243f2025b4e299382e01ac4d4626d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:01.805855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e0f66-87b8-4eb3-be93-0b188000c71c None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:01.806222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e0f66-87b8-4eb3-be93-0b188000c71c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:01.806514 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e0f66-87b8-4eb3-be93-0b188000c71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:01.830248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-c13e0f66-87b8-4eb3-be93-0b188000c71c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:01.831485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c13e0f66-87b8-4eb3-be93-0b188000c71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:01.883405 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:01.896596 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1049/2088] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:01.905628 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d64ca-8a29-4f43-9682-058fb6292a22 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:01.905853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d64ca-8a29-4f43-9682-058fb6292a22 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:01.906125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d64ca-8a29-4f43-9682-058fb6292a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:01.928925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-bb1d64ca-8a29-4f43-9682-058fb6292a22 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:01.930145 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bb1d64ca-8a29-4f43-9682-058fb6292a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:01.939105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1040/2089] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:01.943697 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e0f66-87b8-4eb3-be93-0b188000c71c None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:01.953637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e0f66-87b8-4eb3-be93-0b188000c71c None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:01.959270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d549e88-7778-4bec-a4e2-abae2a5895e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:01.959872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d549e88-7778-4bec-a4e2-abae2a5895e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:01.960150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d549e88-7778-4bec-a4e2-abae2a5895e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:01.960409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d549e88-7778-4bec-a4e2-abae2a5895e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:01.961059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d549e88-7778-4bec-a4e2-abae2a5895e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:33:01.961353 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d549e88-7778-4bec-a4e2-abae2a5895e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:01.963071 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d549e88-7778-4bec-a4e2-abae2a5895e0 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:01.974810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1050/2090] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:33:01.979154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d64ca-8a29-4f43-9682-058fb6292a22 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:01.988804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d64ca-8a29-4f43-9682-058fb6292a22 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:01.994128 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3616a-05a6-4d38-b361-f3bdc918577e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:01.994769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3616a-05a6-4d38-b361-f3bdc918577e admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:01.995041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3616a-05a6-4d38-b361-f3bdc918577e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:01.995299 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3616a-05a6-4d38-b361-f3bdc918577e admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:01.995922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3616a-05a6-4d38-b361-f3bdc918577e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:01.997844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3616a-05a6-4d38-b361-f3bdc918577e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:02.031421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1041/2091] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:33:01 2022] POST /identity/v3/projects => generated 401 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:33:02.036705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d549e88-7778-4bec-a4e2-abae2a5895e0 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:02.046849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d549e88-7778-4bec-a4e2-abae2a5895e0 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:02.110112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2380b1d-0c52-4530-a0eb-85b8a51cddc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:02.110865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2380b1d-0c52-4530-a0eb-85b8a51cddc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:02.111145 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2380b1d-0c52-4530-a0eb-85b8a51cddc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:02.111413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2380b1d-0c52-4530-a0eb-85b8a51cddc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:02.112060 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2380b1d-0c52-4530-a0eb-85b8a51cddc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:02.114264 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2380b1d-0c52-4530-a0eb-85b8a51cddc3 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:02.140035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1051/2092] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:33:02 2022] POST /identity/v3/users => generated 301 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:33:02.141282 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3616a-05a6-4d38-b361-f3bdc918577e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:02.152643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3616a-05a6-4d38-b361-f3bdc918577e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:02.158829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece2209-e858-4cc3-9194-09e5af32643d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:02.159443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece2209-e858-4cc3-9194-09e5af32643d admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:02.159725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece2209-e858-4cc3-9194-09e5af32643d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:02.159992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece2209-e858-4cc3-9194-09e5af32643d admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:02.160675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece2209-e858-4cc3-9194-09e5af32643d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:02.162432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece2209-e858-4cc3-9194-09e5af32643d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:02.174021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1042/2093] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:33:02.178119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2380b1d-0c52-4530-a0eb-85b8a51cddc3 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:02.188402 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2380b1d-0c52-4530-a0eb-85b8a51cddc3 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:02.194063 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0654fb-cf57-4f85-b574-7e4e8df150d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:02.194708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0654fb-cf57-4f85-b574-7e4e8df150d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:02.194984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0654fb-cf57-4f85-b574-7e4e8df150d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:02.195249 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0654fb-cf57-4f85-b574-7e4e8df150d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:02.195894 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0654fb-cf57-4f85-b574-7e4e8df150d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:02.197505 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0654fb-cf57-4f85-b574-7e4e8df150d5 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:02.207810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1052/2094] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:33:02.211824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece2209-e858-4cc3-9194-09e5af32643d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:02.221493 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece2209-e858-4cc3-9194-09e5af32643d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:02.226761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5f6c7-a2d1-4bc6-8948-94cd6048db89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:02.227396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5f6c7-a2d1-4bc6-8948-94cd6048db89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:02.227686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5f6c7-a2d1-4bc6-8948-94cd6048db89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:02.227950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5f6c7-a2d1-4bc6-8948-94cd6048db89 admin admin] PATH_INFO: `/v3/projects/a7f61ae86bf045ad8e6891df24f03624/users/9d65b13aa1f44bb0821f71e9a9b2f6ae/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:02.243812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5f6c7-a2d1-4bc6-8948-94cd6048db89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f61ae86bf045ad8e6891df24f03624, user_id=9d65b13aa1f44bb0821f71e9a9b2f6ae, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:02.246468 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5f6c7-a2d1-4bc6-8948-94cd6048db89 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:02.261299 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1043/2095] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:33:02 2022] PUT /identity/v3/projects/a7f61ae86bf045ad8e6891df24f03624/users/9d65b13aa1f44bb0821f71e9a9b2f6ae/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:33:02.272933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e9260-8951-4eb5-9ab5-012e221f3fc2 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:02.273174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e9260-8951-4eb5-9ab5-012e221f3fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:02.273617 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e9260-8951-4eb5-9ab5-012e221f3fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:02.302210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-a54e9260-8951-4eb5-9ab5-012e221f3fc2 None None] MFA Rules not processed for user `9d65b13aa1f44bb0821f71e9a9b2f6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:02.303515 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a54e9260-8951-4eb5-9ab5-012e221f3fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:02.376174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1053/2096] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:02 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:33:02.380658 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5f6c7-a2d1-4bc6-8948-94cd6048db89 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:02.444471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5f6c7-a2d1-4bc6-8948-94cd6048db89 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:02.449112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39094c4-25cd-4fd8-8009-e723135c73fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:02.449718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39094c4-25cd-4fd8-8009-e723135c73fe admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:02.449991 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39094c4-25cd-4fd8-8009-e723135c73fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:02.450256 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39094c4-25cd-4fd8-8009-e723135c73fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:02.450921 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39094c4-25cd-4fd8-8009-e723135c73fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:02.452816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39094c4-25cd-4fd8-8009-e723135c73fe admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:02.481066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1044/2097] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:33:02 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:33:02.484806 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e9260-8951-4eb5-9ab5-012e221f3fc2 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:02.495755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e9260-8951-4eb5-9ab5-012e221f3fc2 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:02.543900 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:02.554741 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbccd308-5d96-46bf-9085-ca22862dc5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:02.555353 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccd308-5d96-46bf-9085-ca22862dc5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:02.555629 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccd308-5d96-46bf-9085-ca22862dc5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:02.555894 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccd308-5d96-46bf-9085-ca22862dc5c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:02.556521 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbccd308-5d96-46bf-9085-ca22862dc5c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:02.558445 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbccd308-5d96-46bf-9085-ca22862dc5c0 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:02.580317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1054/2098] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:33:02 2022] POST /identity/v3/users => generated 307 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:33:02.584524 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39094c4-25cd-4fd8-8009-e723135c73fe admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:02.594944 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39094c4-25cd-4fd8-8009-e723135c73fe admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:02.599756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58917f-6f50-495f-a2a3-1d2ab7b727e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:02.600354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58917f-6f50-495f-a2a3-1d2ab7b727e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:02.600627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58917f-6f50-495f-a2a3-1d2ab7b727e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:02.600888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58917f-6f50-495f-a2a3-1d2ab7b727e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:02.601495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58917f-6f50-495f-a2a3-1d2ab7b727e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:02.603134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58917f-6f50-495f-a2a3-1d2ab7b727e4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:02.613619 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1045/2099] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:33:02.617976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbccd308-5d96-46bf-9085-ca22862dc5c0 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:02.628778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbccd308-5d96-46bf-9085-ca22862dc5c0 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:02.634614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28830459-23c5-4203-b673-e65b32460fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:02.635274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28830459-23c5-4203-b673-e65b32460fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:02.635557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28830459-23c5-4203-b673-e65b32460fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:02.635834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28830459-23c5-4203-b673-e65b32460fb7 admin admin] PATH_INFO: `/v3/projects/73b481e2ec544c28ad2db1cbb2cb89d3/users/e705494658304fefbe80fe2f0f2009f2/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:02.651248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28830459-23c5-4203-b673-e65b32460fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73b481e2ec544c28ad2db1cbb2cb89d3, user_id=e705494658304fefbe80fe2f0f2009f2, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:02.653924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28830459-23c5-4203-b673-e65b32460fb7 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:02.669328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1055/2100] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:33:02 2022] PUT /identity/v3/projects/73b481e2ec544c28ad2db1cbb2cb89d3/users/e705494658304fefbe80fe2f0f2009f2/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:02.673922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58917f-6f50-495f-a2a3-1d2ab7b727e4 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:02.684274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58917f-6f50-495f-a2a3-1d2ab7b727e4 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:02.696786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:02.742884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af4432-c6a6-4451-95e9-42a7ebfe6cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:02.743500 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af4432-c6a6-4451-95e9-42a7ebfe6cab admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:02.743775 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af4432-c6a6-4451-95e9-42a7ebfe6cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:02.744049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af4432-c6a6-4451-95e9-42a7ebfe6cab admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:02.744694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af4432-c6a6-4451-95e9-42a7ebfe6cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:02.746289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af4432-c6a6-4451-95e9-42a7ebfe6cab admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:02.760109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1046/2101] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:33:02.761335 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28830459-23c5-4203-b673-e65b32460fb7 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:02.772108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28830459-23c5-4203-b673-e65b32460fb7 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:02.777430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd7d2f-61c3-46b3-91a1-a5d09a8196cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:02.778112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd7d2f-61c3-46b3-91a1-a5d09a8196cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:02.778388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd7d2f-61c3-46b3-91a1-a5d09a8196cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:02.778692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd7d2f-61c3-46b3-91a1-a5d09a8196cd admin admin] PATH_INFO: `/v3/projects/73b481e2ec544c28ad2db1cbb2cb89d3/users/e705494658304fefbe80fe2f0f2009f2/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:02.780707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd7d2f-61c3-46b3-91a1-a5d09a8196cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=73b481e2ec544c28ad2db1cbb2cb89d3, user_id=e705494658304fefbe80fe2f0f2009f2, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:02.784011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd7d2f-61c3-46b3-91a1-a5d09a8196cd admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:02.794477 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1056/2102] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:33:02 2022] PUT /identity/v3/projects/73b481e2ec544c28ad2db1cbb2cb89d3/users/e705494658304fefbe80fe2f0f2009f2/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:02.805899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afc0ec-cb12-40d8-b9d3-60670aa8aaf7 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:02.806203 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afc0ec-cb12-40d8-b9d3-60670aa8aaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:02.806470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afc0ec-cb12-40d8-b9d3-60670aa8aaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:02.835495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-28afc0ec-cb12-40d8-b9d3-60670aa8aaf7 None None] MFA Rules not processed for user `e705494658304fefbe80fe2f0f2009f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:02.836713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-28afc0ec-cb12-40d8-b9d3-60670aa8aaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:02.914123 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1047/2103] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:02 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:02.926033 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd7d2f-61c3-46b3-91a1-a5d09a8196cd admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:02.927977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-81fd7d2f-61c3-46b3-91a1-a5d09a8196cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:02.987653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd7d2f-61c3-46b3-91a1-a5d09a8196cd admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:02.991825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cbeb9-06e7-4996-a3a2-a9bed84c0480 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:02.992428 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cbeb9-06e7-4996-a3a2-a9bed84c0480 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:02.992702 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cbeb9-06e7-4996-a3a2-a9bed84c0480 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:02.992979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cbeb9-06e7-4996-a3a2-a9bed84c0480 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:03.002760 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214cbeb9-06e7-4996-a3a2-a9bed84c0480 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:03.004196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214cbeb9-06e7-4996-a3a2-a9bed84c0480 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:03.014161 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1057/2104] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:02 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:33:03.094262 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28afc0ec-cb12-40d8-b9d3-60670aa8aaf7 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:03.095689 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-28afc0ec-cb12-40d8-b9d3-60670aa8aaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:03.108134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28afc0ec-cb12-40d8-b9d3-60670aa8aaf7 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:03.111842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c18a5-a8c0-46f3-b9fe-6f27b3522411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:03.112443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c18a5-a8c0-46f3-b9fe-6f27b3522411 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:03.112719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c18a5-a8c0-46f3-b9fe-6f27b3522411 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:03.112992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c18a5-a8c0-46f3-b9fe-6f27b3522411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:03.170682 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c18a5-a8c0-46f3-b9fe-6f27b3522411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:03.172403 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c18a5-a8c0-46f3-b9fe-6f27b3522411 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:03.182987 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1048/2105] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:03 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:33:08.380399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cbeb9-06e7-4996-a3a2-a9bed84c0480 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:08.445942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cbeb9-06e7-4996-a3a2-a9bed84c0480 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:08.450570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65970e3c-d90c-4ade-be11-f797a777530b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:08.451285 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65970e3c-d90c-4ade-be11-f797a777530b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:08.451619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65970e3c-d90c-4ade-be11-f797a777530b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:08.451953 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65970e3c-d90c-4ade-be11-f797a777530b admin admin] PATH_INFO: `/v3/users/9d65b13aa1f44bb0821f71e9a9b2f6ae` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:08.453333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65970e3c-d90c-4ade-be11-f797a777530b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d65b13aa1f44bb0821f71e9a9b2f6ae)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:08.455455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65970e3c-d90c-4ade-be11-f797a777530b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:08.519329 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-65970e3c-d90c-4ade-be11-f797a777530b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d65b13aa1f44bb0821f71e9a9b2f6ae'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:08.520151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-65970e3c-d90c-4ade-be11-f797a777530b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d65b13aa1f44bb0821f71e9a9b2f6ae'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:08.520786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-65970e3c-d90c-4ade-be11-f797a777530b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d65b13aa1f44bb0821f71e9a9b2f6ae'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:08.522198 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:08.529137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-65970e3c-d90c-4ade-be11-f797a777530b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d65b13aa1f44bb0821f71e9a9b2f6ae'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:08.542225 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-65970e3c-d90c-4ade-be11-f797a777530b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d65b13aa1f44bb0821f71e9a9b2f6ae'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:08.569009 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1058/2106] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:33:08 2022] DELETE /identity/v3/users/9d65b13aa1f44bb0821f71e9a9b2f6ae => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:08.572340 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34add250-819a-4d67-940b-afc7ecd0cb6f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:08.572825 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34add250-819a-4d67-940b-afc7ecd0cb6f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:08.572825 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34add250-819a-4d67-940b-afc7ecd0cb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:08.592409 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-34add250-819a-4d67-940b-afc7ecd0cb6f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:08.593627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-34add250-819a-4d67-940b-afc7ecd0cb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:08.627584 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:08.655216 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1049/2107] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:08.665014 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65970e3c-d90c-4ade-be11-f797a777530b admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:08.666516 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-65970e3c-d90c-4ade-be11-f797a777530b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:08.731714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65970e3c-d90c-4ade-be11-f797a777530b admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:08.736291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b81e9-6385-4454-88f3-b9d622dbaf84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:08.736987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b81e9-6385-4454-88f3-b9d622dbaf84 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:08.737273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b81e9-6385-4454-88f3-b9d622dbaf84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:08.737539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b81e9-6385-4454-88f3-b9d622dbaf84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:08.747654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b81e9-6385-4454-88f3-b9d622dbaf84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:08.749093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b81e9-6385-4454-88f3-b9d622dbaf84 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:08.759970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1059/2108] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:33:09.043108 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34add250-819a-4d67-940b-afc7ecd0cb6f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:09.044778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-34add250-819a-4d67-940b-afc7ecd0cb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:09.058132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34add250-819a-4d67-940b-afc7ecd0cb6f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:09.063245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:09.063718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:09.064067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:09.064412 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] PATH_INFO: `/v3/projects/a7f61ae86bf045ad8e6891df24f03624` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:09.065839 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7f61ae86bf045ad8e6891df24f03624)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:09.067976 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:09.126922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7f61ae86bf045ad8e6891df24f03624', 'request_id': 'req-89fd491c-6d05-491c-8af9-b1d76d8ea848'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.133042 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7f61ae86bf045ad8e6891df24f03624', 'request_id': 'req-89fd491c-6d05-491c-8af9-b1d76d8ea848'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.138553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7f61ae86bf045ad8e6891df24f03624', 'request_id': 'req-89fd491c-6d05-491c-8af9-b1d76d8ea848'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.151888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] The token cache is being invalidate because project a7f61ae86bf045ad8e6891df24f03624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:33:09.152193 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.152805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.154288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1050/2109] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:33:09 2022] DELETE /identity/v3/projects/a7f61ae86bf045ad8e6891df24f03624 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:09.158402 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b81e9-6385-4454-88f3-b9d622dbaf84 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:09.168383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b81e9-6385-4454-88f3-b9d622dbaf84 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:09.223651 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d468be-0e0a-4c3e-90fc-4f8cae99e89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:09.224310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d468be-0e0a-4c3e-90fc-4f8cae99e89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:09.224584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d468be-0e0a-4c3e-90fc-4f8cae99e89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:09.224853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d468be-0e0a-4c3e-90fc-4f8cae99e89a admin admin] PATH_INFO: `/v3/users/e705494658304fefbe80fe2f0f2009f2` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:09.225980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d468be-0e0a-4c3e-90fc-4f8cae99e89a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e705494658304fefbe80fe2f0f2009f2)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:09.227943 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d468be-0e0a-4c3e-90fc-4f8cae99e89a admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:09.278598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-94d468be-0e0a-4c3e-90fc-4f8cae99e89a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e705494658304fefbe80fe2f0f2009f2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.279414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-94d468be-0e0a-4c3e-90fc-4f8cae99e89a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e705494658304fefbe80fe2f0f2009f2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.280307 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-94d468be-0e0a-4c3e-90fc-4f8cae99e89a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e705494658304fefbe80fe2f0f2009f2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.288246 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-94d468be-0e0a-4c3e-90fc-4f8cae99e89a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e705494658304fefbe80fe2f0f2009f2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.304249 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:09.306565 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-94d468be-0e0a-4c3e-90fc-4f8cae99e89a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e705494658304fefbe80fe2f0f2009f2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.324222 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1060/2110] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:33:09 2022] DELETE /identity/v3/users/e705494658304fefbe80fe2f0f2009f2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:09.608240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:09.609747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:09.622835 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd491c-6d05-491c-8af9-b1d76d8ea848 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:09.683739 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:09.684449 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:09.684728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:09.684997 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2 admin admin] PATH_INFO: `/v3/projects/73b481e2ec544c28ad2db1cbb2cb89d3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:09.686116 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73b481e2ec544c28ad2db1cbb2cb89d3)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:09.688064 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:09.735143 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73b481e2ec544c28ad2db1cbb2cb89d3', 'request_id': 'req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.741095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73b481e2ec544c28ad2db1cbb2cb89d3', 'request_id': 'req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.746413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73b481e2ec544c28ad2db1cbb2cb89d3', 'request_id': 'req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.748035 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:09.761714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2 admin admin] The token cache is being invalidate because project 73b481e2ec544c28ad2db1cbb2cb89d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:33:09.761997 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.762615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-ec5ccf36-2c2a-47d1-97a9-ca28db1201e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:09.764125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1051/2111] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:33:09 2022] DELETE /identity/v3/projects/73b481e2ec544c28ad2db1cbb2cb89d3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:09.769247 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae0bbe-5c61-45cd-bb1d-f0415b0adde2 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:09.769779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae0bbe-5c61-45cd-bb1d-f0415b0adde2 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:09.769779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae0bbe-5c61-45cd-bb1d-f0415b0adde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:09.790301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-74ae0bbe-5c61-45cd-bb1d-f0415b0adde2 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:09.791539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-74ae0bbe-5c61-45cd-bb1d-f0415b0adde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:09.852179 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1061/2112] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:09.863747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d8c30-65a3-4c31-88fa-1c4fa771e07d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:09.864051 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d8c30-65a3-4c31-88fa-1c4fa771e07d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:09.864312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d8c30-65a3-4c31-88fa-1c4fa771e07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:09.885129 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8a2d8c30-65a3-4c31-88fa-1c4fa771e07d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:09.886349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8a2d8c30-65a3-4c31-88fa-1c4fa771e07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:09.895878 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1052/2113] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:09.901119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae0bbe-5c61-45cd-bb1d-f0415b0adde2 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:09.914580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae0bbe-5c61-45cd-bb1d-f0415b0adde2 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:09.921912 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c5e2c-632c-4fa3-af06-2f23e3ceb550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:09.923108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c5e2c-632c-4fa3-af06-2f23e3ceb550 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:09.923408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c5e2c-632c-4fa3-af06-2f23e3ceb550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:09.923666 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c5e2c-632c-4fa3-af06-2f23e3ceb550 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:09.924573 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c5e2c-632c-4fa3-af06-2f23e3ceb550 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:33:09.924937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c5e2c-632c-4fa3-af06-2f23e3ceb550 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:09.926977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c5e2c-632c-4fa3-af06-2f23e3ceb550 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:09.938655 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1062/2114] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:33:09.943332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d8c30-65a3-4c31-88fa-1c4fa771e07d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:09.952890 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d8c30-65a3-4c31-88fa-1c4fa771e07d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:09.958633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b70f8-48d7-4127-b268-fa1f621261b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:09.959314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b70f8-48d7-4127-b268-fa1f621261b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:09.959592 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b70f8-48d7-4127-b268-fa1f621261b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:09.959854 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b70f8-48d7-4127-b268-fa1f621261b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:09.960484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b70f8-48d7-4127-b268-fa1f621261b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:09.962422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b70f8-48d7-4127-b268-fa1f621261b8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:09.993955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1053/2115] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:33:09 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:33:09.998110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c5e2c-632c-4fa3-af06-2f23e3ceb550 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:10.008108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c5e2c-632c-4fa3-af06-2f23e3ceb550 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:10.065220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b81b3b-1255-44a2-94c6-8f6a76eeab74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:10.065745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b81b3b-1255-44a2-94c6-8f6a76eeab74 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:10.066011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b81b3b-1255-44a2-94c6-8f6a76eeab74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:10.066260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b81b3b-1255-44a2-94c6-8f6a76eeab74 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:10.066910 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b81b3b-1255-44a2-94c6-8f6a76eeab74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:10.068823 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b81b3b-1255-44a2-94c6-8f6a76eeab74 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:10.092194 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1063/2116] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:33:09 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:33:10.096480 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b70f8-48d7-4127-b268-fa1f621261b8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:10.107577 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b70f8-48d7-4127-b268-fa1f621261b8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:10.113057 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f3142-5acd-48b6-8dc3-760cca815de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:10.113657 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f3142-5acd-48b6-8dc3-760cca815de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:10.113933 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f3142-5acd-48b6-8dc3-760cca815de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:10.114201 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f3142-5acd-48b6-8dc3-760cca815de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:10.115043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f3142-5acd-48b6-8dc3-760cca815de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:10.116614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f3142-5acd-48b6-8dc3-760cca815de4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:10.127031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1054/2117] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:33:10.131366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b81b3b-1255-44a2-94c6-8f6a76eeab74 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:10.142416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b81b3b-1255-44a2-94c6-8f6a76eeab74 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:10.148772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a80730-4524-4364-9e35-2881ed2d11e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:10.149391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a80730-4524-4364-9e35-2881ed2d11e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:10.149668 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a80730-4524-4364-9e35-2881ed2d11e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:10.149935 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a80730-4524-4364-9e35-2881ed2d11e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:10.150613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a80730-4524-4364-9e35-2881ed2d11e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:10.152208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a80730-4524-4364-9e35-2881ed2d11e2 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:10.164169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1064/2118] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:33:10.170992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f3142-5acd-48b6-8dc3-760cca815de4 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:10.185943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f3142-5acd-48b6-8dc3-760cca815de4 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:10.193158 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c2aec-7b6b-483f-a2d4-32727dcf63fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:10.193806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c2aec-7b6b-483f-a2d4-32727dcf63fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:10.194085 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c2aec-7b6b-483f-a2d4-32727dcf63fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:10.194358 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c2aec-7b6b-483f-a2d4-32727dcf63fd admin admin] PATH_INFO: `/v3/projects/f6bf5409508042ecbfa5a91ec8880d96/users/819f7e42956747d98fa0e6ed8d5b9bc8/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:10.208674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c2aec-7b6b-483f-a2d4-32727dcf63fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6bf5409508042ecbfa5a91ec8880d96, user_id=819f7e42956747d98fa0e6ed8d5b9bc8, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:10.211399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c2aec-7b6b-483f-a2d4-32727dcf63fd admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:10.226769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1055/2119] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:33:10 2022] PUT /identity/v3/projects/f6bf5409508042ecbfa5a91ec8880d96/users/819f7e42956747d98fa0e6ed8d5b9bc8/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:10.238224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b0b31-3a86-428d-9332-05f79d952711 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:10.238524 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b0b31-3a86-428d-9332-05f79d952711 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:10.238808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b0b31-3a86-428d-9332-05f79d952711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:10.267670 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-c86b0b31-3a86-428d-9332-05f79d952711 None None] MFA Rules not processed for user `819f7e42956747d98fa0e6ed8d5b9bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:10.268882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c86b0b31-3a86-428d-9332-05f79d952711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:10.310546 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:10.335460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1065/2120] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:10.339984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c2aec-7b6b-483f-a2d4-32727dcf63fd admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:10.403054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c2aec-7b6b-483f-a2d4-32727dcf63fd admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:10.408329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac50ff0-afc2-4c2a-a8a2-e979061d4697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:10.408936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac50ff0-afc2-4c2a-a8a2-e979061d4697 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:10.409208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac50ff0-afc2-4c2a-a8a2-e979061d4697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:10.409477 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac50ff0-afc2-4c2a-a8a2-e979061d4697 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:10.410131 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac50ff0-afc2-4c2a-a8a2-e979061d4697 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:10.412141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac50ff0-afc2-4c2a-a8a2-e979061d4697 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:10.441080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1056/2121] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:33:10 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:33:10.445359 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b0b31-3a86-428d-9332-05f79d952711 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:10.456469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b0b31-3a86-428d-9332-05f79d952711 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:10.515740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddcbdb-2584-4f78-abea-09f180b93354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:10.516492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddcbdb-2584-4f78-abea-09f180b93354 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:10.516837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddcbdb-2584-4f78-abea-09f180b93354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:10.517170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddcbdb-2584-4f78-abea-09f180b93354 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:10.517926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddcbdb-2584-4f78-abea-09f180b93354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:10.519987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddcbdb-2584-4f78-abea-09f180b93354 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:10.547269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1066/2122] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:33:10 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:33:10.551440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac50ff0-afc2-4c2a-a8a2-e979061d4697 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:10.562451 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac50ff0-afc2-4c2a-a8a2-e979061d4697 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:10.567158 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd2f76-2da4-4204-b6fa-2886774d0a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:10.567873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd2f76-2da4-4204-b6fa-2886774d0a16 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:10.568841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd2f76-2da4-4204-b6fa-2886774d0a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:10.569147 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd2f76-2da4-4204-b6fa-2886774d0a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:10.569796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd2f76-2da4-4204-b6fa-2886774d0a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:10.571442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd2f76-2da4-4204-b6fa-2886774d0a16 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:10.585369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1057/2123] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:33:10.586648 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddcbdb-2584-4f78-abea-09f180b93354 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:10.596684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddcbdb-2584-4f78-abea-09f180b93354 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:10.601483 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7098e5-13ba-4df4-92b6-9d1f20262c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:10.602295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7098e5-13ba-4df4-92b6-9d1f20262c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:10.602688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7098e5-13ba-4df4-92b6-9d1f20262c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:10.602961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7098e5-13ba-4df4-92b6-9d1f20262c12 admin admin] PATH_INFO: `/v3/projects/ee57b45ede654d5aa20ad61fcd99c49a/users/f1c2a36d808e42459d6a941241f7a53a/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:10.618577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7098e5-13ba-4df4-92b6-9d1f20262c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee57b45ede654d5aa20ad61fcd99c49a, user_id=f1c2a36d808e42459d6a941241f7a53a, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:10.621732 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7098e5-13ba-4df4-92b6-9d1f20262c12 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:10.636533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1067/2124] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:33:10 2022] PUT /identity/v3/projects/ee57b45ede654d5aa20ad61fcd99c49a/users/f1c2a36d808e42459d6a941241f7a53a/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:10.640682 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd2f76-2da4-4204-b6fa-2886774d0a16 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:10.653259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd2f76-2da4-4204-b6fa-2886774d0a16 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:10.714227 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7c54f-281d-4f17-8726-7c7ce0a35cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:10.714866 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7c54f-281d-4f17-8726-7c7ce0a35cec admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:10.715141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7c54f-281d-4f17-8726-7c7ce0a35cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:10.715410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7c54f-281d-4f17-8726-7c7ce0a35cec admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:10.716056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7c54f-281d-4f17-8726-7c7ce0a35cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:10.717638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7c54f-281d-4f17-8726-7c7ce0a35cec admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:10.719802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:10.729485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1058/2125] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:33:10.745191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7098e5-13ba-4df4-92b6-9d1f20262c12 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:10.755360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7098e5-13ba-4df4-92b6-9d1f20262c12 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:10.759980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12802ba2-4631-430c-9887-12a8a10f1f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:10.760629 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12802ba2-4631-430c-9887-12a8a10f1f28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:10.760903 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12802ba2-4631-430c-9887-12a8a10f1f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:10.761176 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12802ba2-4631-430c-9887-12a8a10f1f28 admin admin] PATH_INFO: `/v3/projects/ee57b45ede654d5aa20ad61fcd99c49a/users/f1c2a36d808e42459d6a941241f7a53a/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:10.763064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12802ba2-4631-430c-9887-12a8a10f1f28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee57b45ede654d5aa20ad61fcd99c49a, user_id=f1c2a36d808e42459d6a941241f7a53a, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:10.765690 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12802ba2-4631-430c-9887-12a8a10f1f28 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:10.776467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1068/2126] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:33:10 2022] PUT /identity/v3/projects/ee57b45ede654d5aa20ad61fcd99c49a/users/f1c2a36d808e42459d6a941241f7a53a/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:10.786984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c4cad-6496-4ccd-a476-1e020c352802 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:10.787266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c4cad-6496-4ccd-a476-1e020c352802 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:10.787535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c4cad-6496-4ccd-a476-1e020c352802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:10.815191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-582c4cad-6496-4ccd-a476-1e020c352802 None None] MFA Rules not processed for user `f1c2a36d808e42459d6a941241f7a53a`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:10.816405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-582c4cad-6496-4ccd-a476-1e020c352802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:10.891431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1059/2127] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:10.902607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12802ba2-4631-430c-9887-12a8a10f1f28 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:10.904404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-12802ba2-4631-430c-9887-12a8a10f1f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:10.963717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12802ba2-4631-430c-9887-12a8a10f1f28 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:10.967704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c5d52-1e16-4ea5-a563-1049b917b58e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:10.968306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c5d52-1e16-4ea5-a563-1049b917b58e service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:10.968583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c5d52-1e16-4ea5-a563-1049b917b58e service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:10.968852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c5d52-1e16-4ea5-a563-1049b917b58e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:10.977946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c5d52-1e16-4ea5-a563-1049b917b58e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:10.979403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c5d52-1e16-4ea5-a563-1049b917b58e service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:10.989158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1069/2128] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:33:11.092793 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c4cad-6496-4ccd-a476-1e020c352802 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:11.103352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c4cad-6496-4ccd-a476-1e020c352802 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:11.107624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e66497-4698-46ae-bc1b-2b6e76bc20f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:11.108234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66497-4698-46ae-bc1b-2b6e76bc20f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:11.108515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66497-4698-46ae-bc1b-2b6e76bc20f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:11.108791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66497-4698-46ae-bc1b-2b6e76bc20f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:11.166031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e66497-4698-46ae-bc1b-2b6e76bc20f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:11.167534 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e66497-4698-46ae-bc1b-2b6e76bc20f1 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:11.176989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1060/2129] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:33:13.803694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c5d52-1e16-4ea5-a563-1049b917b58e service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:13.845908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:13.868984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c5d52-1e16-4ea5-a563-1049b917b58e service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:13.874759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda2bed-0c16-4fab-9e33-f8012cc2332e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:13.875395 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda2bed-0c16-4fab-9e33-f8012cc2332e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:13.875671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda2bed-0c16-4fab-9e33-f8012cc2332e admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:13.875941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda2bed-0c16-4fab-9e33-f8012cc2332e admin admin] PATH_INFO: `/v3/users/819f7e42956747d98fa0e6ed8d5b9bc8` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:13.877183 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda2bed-0c16-4fab-9e33-f8012cc2332e admin admin] RBAC: Authorizing `identity:delete_user(user_id=819f7e42956747d98fa0e6ed8d5b9bc8)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:13.879168 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda2bed-0c16-4fab-9e33-f8012cc2332e admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:13.927737 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-eeda2bed-0c16-4fab-9e33-f8012cc2332e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '819f7e42956747d98fa0e6ed8d5b9bc8'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:13.928296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-eeda2bed-0c16-4fab-9e33-f8012cc2332e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '819f7e42956747d98fa0e6ed8d5b9bc8'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:13.928880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-eeda2bed-0c16-4fab-9e33-f8012cc2332e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '819f7e42956747d98fa0e6ed8d5b9bc8'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:13.936409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-eeda2bed-0c16-4fab-9e33-f8012cc2332e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '819f7e42956747d98fa0e6ed8d5b9bc8'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:13.948665 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-eeda2bed-0c16-4fab-9e33-f8012cc2332e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '819f7e42956747d98fa0e6ed8d5b9bc8'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:13.962739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1070/2130] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:33:13 2022] DELETE /identity/v3/users/819f7e42956747d98fa0e6ed8d5b9bc8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:13.967435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d015bb-c929-4169-b306-9be542da8d66 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:13.968032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d015bb-c929-4169-b306-9be542da8d66 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:13.968032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d015bb-c929-4169-b306-9be542da8d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:13.987961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-28d015bb-c929-4169-b306-9be542da8d66 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:13.989146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-28d015bb-c929-4169-b306-9be542da8d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:14.052760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1061/2131] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:14.063363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda2bed-0c16-4fab-9e33-f8012cc2332e admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:14.064999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-eeda2bed-0c16-4fab-9e33-f8012cc2332e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:14.128450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda2bed-0c16-4fab-9e33-f8012cc2332e admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:14.132432 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393ea72-601a-4625-82a7-1cb78c9efdb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:14.133044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393ea72-601a-4625-82a7-1cb78c9efdb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:14.133332 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393ea72-601a-4625-82a7-1cb78c9efdb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:14.133681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393ea72-601a-4625-82a7-1cb78c9efdb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:14.143647 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393ea72-601a-4625-82a7-1cb78c9efdb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:14.145075 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393ea72-601a-4625-82a7-1cb78c9efdb6 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:14.155262 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1071/2132] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:33:14.513640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d015bb-c929-4169-b306-9be542da8d66 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:14.515121 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-28d015bb-c929-4169-b306-9be542da8d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:14.530040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d015bb-c929-4169-b306-9be542da8d66 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:14.536915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:14.537424 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:14.538059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:14.538059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] PATH_INFO: `/v3/projects/f6bf5409508042ecbfa5a91ec8880d96` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:14.539270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6bf5409508042ecbfa5a91ec8880d96)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:14.541601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:14.551875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393ea72-601a-4625-82a7-1cb78c9efdb6 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:14.553537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d393ea72-601a-4625-82a7-1cb78c9efdb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:14.570609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:14.574465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393ea72-601a-4625-82a7-1cb78c9efdb6 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:14.579627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b23482-f7cf-46ba-8039-4eb9743e3a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:14.580350 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b23482-f7cf-46ba-8039-4eb9743e3a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:14.580647 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b23482-f7cf-46ba-8039-4eb9743e3a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:14.580940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b23482-f7cf-46ba-8039-4eb9743e3a4b admin admin] PATH_INFO: `/v3/users/2f9f213cb055481cbd5b018acbd0cde5` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:14.582470 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b23482-f7cf-46ba-8039-4eb9743e3a4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9f213cb055481cbd5b018acbd0cde5)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:14.585078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b23482-f7cf-46ba-8039-4eb9743e3a4b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:14.608471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6bf5409508042ecbfa5a91ec8880d96', 'request_id': 'req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.615869 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6bf5409508042ecbfa5a91ec8880d96', 'request_id': 'req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.624351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6bf5409508042ecbfa5a91ec8880d96', 'request_id': 'req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.643110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] The token cache is being invalidate because project f6bf5409508042ecbfa5a91ec8880d96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:33:14.643110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.643110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.648179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1062/2133] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:33:14 2022] DELETE /identity/v3/projects/f6bf5409508042ecbfa5a91ec8880d96 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:14.648311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-13b23482-f7cf-46ba-8039-4eb9743e3a4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9f213cb055481cbd5b018acbd0cde5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.648311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-13b23482-f7cf-46ba-8039-4eb9743e3a4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9f213cb055481cbd5b018acbd0cde5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.648311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-13b23482-f7cf-46ba-8039-4eb9743e3a4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9f213cb055481cbd5b018acbd0cde5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.649267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:14.650711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:14.654703 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:14.657184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-13b23482-f7cf-46ba-8039-4eb9743e3a4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9f213cb055481cbd5b018acbd0cde5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.674702 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-13b23482-f7cf-46ba-8039-4eb9743e3a4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9f213cb055481cbd5b018acbd0cde5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.690112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1072/2134] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:33:14 2022] DELETE /identity/v3/users/2f9f213cb055481cbd5b018acbd0cde5 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:33:14.695166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d54601-06b6-4e62-979d-829769194571 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:14.695571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d54601-06b6-4e62-979d-829769194571 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:14.695921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d54601-06b6-4e62-979d-829769194571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:14.718780 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-b1d54601-06b6-4e62-979d-829769194571 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:14.720272 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b1d54601-06b6-4e62-979d-829769194571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:14.736017 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfab73-144b-4905-9f56-a3da0ab0fc9b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:14.822581 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0b82f-384d-4e1d-9102-6d57f31b636b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:14.823278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0b82f-384d-4e1d-9102-6d57f31b636b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:14.823600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0b82f-384d-4e1d-9102-6d57f31b636b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:14.824520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0b82f-384d-4e1d-9102-6d57f31b636b admin admin] PATH_INFO: `/v3/users/f1c2a36d808e42459d6a941241f7a53a` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:14.825344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1073/2135] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:14.825789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0b82f-384d-4e1d-9102-6d57f31b636b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1c2a36d808e42459d6a941241f7a53a)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:14.827721 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0b82f-384d-4e1d-9102-6d57f31b636b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:14.833955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d54601-06b6-4e62-979d-829769194571 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:14.837740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b1d54601-06b6-4e62-979d-829769194571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:14.895248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0fc0b82f-384d-4e1d-9102-6d57f31b636b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1c2a36d808e42459d6a941241f7a53a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.896156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0fc0b82f-384d-4e1d-9102-6d57f31b636b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1c2a36d808e42459d6a941241f7a53a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.896856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0fc0b82f-384d-4e1d-9102-6d57f31b636b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1c2a36d808e42459d6a941241f7a53a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.908364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:14.910059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d54601-06b6-4e62-979d-829769194571 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:14.911209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0fc0b82f-384d-4e1d-9102-6d57f31b636b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1c2a36d808e42459d6a941241f7a53a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.914884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464a6cf-5767-489f-8541-84fb840f1ba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:14.917209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464a6cf-5767-489f-8541-84fb840f1ba1 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:14.917486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464a6cf-5767-489f-8541-84fb840f1ba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:14.917763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464a6cf-5767-489f-8541-84fb840f1ba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:14.920926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f464a6cf-5767-489f-8541-84fb840f1ba1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:14.929675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0fc0b82f-384d-4e1d-9102-6d57f31b636b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1c2a36d808e42459d6a941241f7a53a'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:14.937972 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464a6cf-5767-489f-8541-84fb840f1ba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:14.939598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464a6cf-5767-489f-8541-84fb840f1ba1 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:14.949518 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1063/2136] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:33:14 2022] DELETE /identity/v3/users/f1c2a36d808e42459d6a941241f7a53a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:33:14.952811 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1074/2137] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:33:15.380088 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0b82f-384d-4e1d-9102-6d57f31b636b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:15.382995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0fc0b82f-384d-4e1d-9102-6d57f31b636b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:15.437753 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464a6cf-5767-489f-8541-84fb840f1ba1 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:15.439569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f464a6cf-5767-489f-8541-84fb840f1ba1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:15.472171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464a6cf-5767-489f-8541-84fb840f1ba1 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:15.499018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0b82f-384d-4e1d-9102-6d57f31b636b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:15.503990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:15.504493 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:15.504750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:15.505397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160 admin admin] PATH_INFO: `/v3/projects/d862d97a5aaa4ea3976f7d7768cd45b4` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:15.507365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d862d97a5aaa4ea3976f7d7768cd45b4)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:15.509517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:15.606299 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd862d97a5aaa4ea3976f7d7768cd45b4', 'request_id': 'req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:15.606299 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd862d97a5aaa4ea3976f7d7768cd45b4', 'request_id': 'req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:15.608602 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b329e-31be-40bd-8319-ba2287dcca28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:15.609263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b329e-31be-40bd-8319-ba2287dcca28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:15.609569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b329e-31be-40bd-8319-ba2287dcca28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:15.609837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b329e-31be-40bd-8319-ba2287dcca28 admin admin] PATH_INFO: `/v3/projects/ee57b45ede654d5aa20ad61fcd99c49a` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:15.611061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b329e-31be-40bd-8319-ba2287dcca28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee57b45ede654d5aa20ad61fcd99c49a)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:15.614068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b329e-31be-40bd-8319-ba2287dcca28 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:15.615114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd862d97a5aaa4ea3976f7d7768cd45b4', 'request_id': 'req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:15.635673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160 admin admin] The token cache is being invalidate because project d862d97a5aaa4ea3976f7d7768cd45b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:33:15.637665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:15.639636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a7fb047d-8ed0-43b7-9817-80ec7a5e9160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:15.642589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1064/2138] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:33:15 2022] DELETE /identity/v3/projects/d862d97a5aaa4ea3976f7d7768cd45b4 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:15.649065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:15.658225 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5be9e-925a-411d-9af7-3e9a39f93dde None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:15.660103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5be9e-925a-411d-9af7-3e9a39f93dde None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:15.662227 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5be9e-925a-411d-9af7-3e9a39f93dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:15.701263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b8b5be9e-925a-411d-9af7-3e9a39f93dde None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:15.702844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b8b5be9e-925a-411d-9af7-3e9a39f93dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:15.727082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1b6b329e-31be-40bd-8319-ba2287dcca28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee57b45ede654d5aa20ad61fcd99c49a', 'request_id': 'req-1b6b329e-31be-40bd-8319-ba2287dcca28'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:15.748726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1b6b329e-31be-40bd-8319-ba2287dcca28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee57b45ede654d5aa20ad61fcd99c49a', 'request_id': 'req-1b6b329e-31be-40bd-8319-ba2287dcca28'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:15.768660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1b6b329e-31be-40bd-8319-ba2287dcca28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee57b45ede654d5aa20ad61fcd99c49a', 'request_id': 'req-1b6b329e-31be-40bd-8319-ba2287dcca28'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:15.784829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1b6b329e-31be-40bd-8319-ba2287dcca28 admin admin] The token cache is being invalidate because project ee57b45ede654d5aa20ad61fcd99c49a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:33:15.785188 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1b6b329e-31be-40bd-8319-ba2287dcca28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:15.785380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1065/2139] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:15.785910 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1b6b329e-31be-40bd-8319-ba2287dcca28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:15.787660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1075/2140] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:33:15 2022] DELETE /identity/v3/projects/ee57b45ede654d5aa20ad61fcd99c49a => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:15.793735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1636e-c01c-451d-b190-e486bd9a74b3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:15.794026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1636e-c01c-451d-b190-e486bd9a74b3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:15.794300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1636e-c01c-451d-b190-e486bd9a74b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:15.795006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01d280-0124-47e6-abcf-cf7364da1e8d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:15.795418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01d280-0124-47e6-abcf-cf7364da1e8d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:15.795781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01d280-0124-47e6-abcf-cf7364da1e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:15.803275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:15.817415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-ca01d280-0124-47e6-abcf-cf7364da1e8d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:15.819024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ca01d280-0124-47e6-abcf-cf7364da1e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:15.827981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1076/2141] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:33:15.828880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e0a1636e-c01c-451d-b190-e486bd9a74b3 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:15.830790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e0a1636e-c01c-451d-b190-e486bd9a74b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:15.833403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01d280-0124-47e6-abcf-cf7364da1e8d None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:15.844261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1066/2142] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:15.847063 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01d280-0124-47e6-abcf-cf7364da1e8d None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:15.851869 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703325d-475c-44d1-ba8d-cd96d38d9968 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:15.852178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703325d-475c-44d1-ba8d-cd96d38d9968 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:15.853065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703325d-475c-44d1-ba8d-cd96d38d9968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:15.857978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fd2d0-8400-412d-adeb-dedf6dad8cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:15.858861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fd2d0-8400-412d-adeb-dedf6dad8cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:15.859282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fd2d0-8400-412d-adeb-dedf6dad8cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:15.859715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fd2d0-8400-412d-adeb-dedf6dad8cc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:15.860532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fd2d0-8400-412d-adeb-dedf6dad8cc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:33:15.860924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fd2d0-8400-412d-adeb-dedf6dad8cc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:15.862854 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fd2d0-8400-412d-adeb-dedf6dad8cc7 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:15.879427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5703325d-475c-44d1-ba8d-cd96d38d9968 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:15.881607 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5703325d-475c-44d1-ba8d-cd96d38d9968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:15.893124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1067/2143] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:15.896569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703325d-475c-44d1-ba8d-cd96d38d9968 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:15.899821 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1077/2144] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:33:15.900874 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fd2d0-8400-412d-adeb-dedf6dad8cc7 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:15.907022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703325d-475c-44d1-ba8d-cd96d38d9968 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:15.913036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347cfaf2-9b45-4616-a0ce-f0312c53d6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:15.913666 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cfaf2-9b45-4616-a0ce-f0312c53d6ab admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:15.913941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cfaf2-9b45-4616-a0ce-f0312c53d6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:15.914529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cfaf2-9b45-4616-a0ce-f0312c53d6ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:15.914889 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fd2d0-8400-412d-adeb-dedf6dad8cc7 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:15.916339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347cfaf2-9b45-4616-a0ce-f0312c53d6ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:33:15.916644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347cfaf2-9b45-4616-a0ce-f0312c53d6ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:15.919306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347cfaf2-9b45-4616-a0ce-f0312c53d6ab admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:15.922933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78c2dc-fcb1-444f-8a9b-4b7e7f92d5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:15.923788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78c2dc-fcb1-444f-8a9b-4b7e7f92d5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:15.924204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78c2dc-fcb1-444f-8a9b-4b7e7f92d5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:15.924601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78c2dc-fcb1-444f-8a9b-4b7e7f92d5b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:15.925426 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78c2dc-fcb1-444f-8a9b-4b7e7f92d5b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:15.928274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78c2dc-fcb1-444f-8a9b-4b7e7f92d5b1 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:15.942735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1068/2145] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:33:15.949280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347cfaf2-9b45-4616-a0ce-f0312c53d6ab admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:15.961315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347cfaf2-9b45-4616-a0ce-f0312c53d6ab admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:15.969050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad214a-11b9-4f06-b2cf-807900bb7a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:15.969932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad214a-11b9-4f06-b2cf-807900bb7a05 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:15.970471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad214a-11b9-4f06-b2cf-807900bb7a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:15.971081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad214a-11b9-4f06-b2cf-807900bb7a05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:15.972015 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad214a-11b9-4f06-b2cf-807900bb7a05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:15.974013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad214a-11b9-4f06-b2cf-807900bb7a05 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:15.982575 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1078/2146] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:33:15 2022] POST /identity/v3/projects => generated 375 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:33:15.987770 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78c2dc-fcb1-444f-8a9b-4b7e7f92d5b1 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.008815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1069/2147] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:33:15 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:33:16.013075 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad214a-11b9-4f06-b2cf-807900bb7a05 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.058981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78c2dc-fcb1-444f-8a9b-4b7e7f92d5b1 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.074847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad214a-11b9-4f06-b2cf-807900bb7a05 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.080243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ec1e8-ede4-4842-8f8d-a6eda5894047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.080860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ec1e8-ede4-4842-8f8d-a6eda5894047 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.081141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ec1e8-ede4-4842-8f8d-a6eda5894047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.081415 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ec1e8-ede4-4842-8f8d-a6eda5894047 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.082102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ec1e8-ede4-4842-8f8d-a6eda5894047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:16.084133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ec1e8-ede4-4842-8f8d-a6eda5894047 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:16.105322 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1070/2148] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:33:16 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:33:16.109845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ec1e8-ede4-4842-8f8d-a6eda5894047 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.121086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ec1e8-ede4-4842-8f8d-a6eda5894047 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.124541 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd0f24-a792-433d-8b91-450de599ff5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.125390 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd0f24-a792-433d-8b91-450de599ff5f admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.125776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd0f24-a792-433d-8b91-450de599ff5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.126108 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167bc7f-c2a2-42cb-a442-2796a7493995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.126221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd0f24-a792-433d-8b91-450de599ff5f admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.126756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167bc7f-c2a2-42cb-a442-2796a7493995 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.127030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167bc7f-c2a2-42cb-a442-2796a7493995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.127123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd0f24-a792-433d-8b91-450de599ff5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:16.127320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167bc7f-c2a2-42cb-a442-2796a7493995 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.127966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167bc7f-c2a2-42cb-a442-2796a7493995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:16.129606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167bc7f-c2a2-42cb-a442-2796a7493995 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:16.130417 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd0f24-a792-433d-8b91-450de599ff5f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:16.143664 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1071/2149] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:33:16.144725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167bc7f-c2a2-42cb-a442-2796a7493995 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.150300 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:16.156837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167bc7f-c2a2-42cb-a442-2796a7493995 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.160087 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1079/2150] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:33:15 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:33:16.164381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd0f24-a792-433d-8b91-450de599ff5f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.165230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d9189-d5ec-417b-a4b8-4e087d5791c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.165840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d9189-d5ec-417b-a4b8-4e087d5791c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.166115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d9189-d5ec-417b-a4b8-4e087d5791c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.166380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d9189-d5ec-417b-a4b8-4e087d5791c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.167076 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d9189-d5ec-417b-a4b8-4e087d5791c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:16.168690 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d9189-d5ec-417b-a4b8-4e087d5791c7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:16.175054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd0f24-a792-433d-8b91-450de599ff5f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.179716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13af018-1e54-473c-8533-b7d006345e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.179978 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1072/2151] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:33:16.180314 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13af018-1e54-473c-8533-b7d006345e2f admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.180631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13af018-1e54-473c-8533-b7d006345e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.180863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13af018-1e54-473c-8533-b7d006345e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.181509 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13af018-1e54-473c-8533-b7d006345e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:16.183410 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13af018-1e54-473c-8533-b7d006345e2f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:16.184063 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d9189-d5ec-417b-a4b8-4e087d5791c7 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.193511 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d9189-d5ec-417b-a4b8-4e087d5791c7 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.200511 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1080/2152] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:33:16.200677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99918634-3e6b-4a73-83a5-4f8fa3b3d11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.200677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99918634-3e6b-4a73-83a5-4f8fa3b3d11a admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.200677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99918634-3e6b-4a73-83a5-4f8fa3b3d11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.200677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99918634-3e6b-4a73-83a5-4f8fa3b3d11a admin admin] PATH_INFO: `/v3/projects/e34ddb26f3df4ac1980df6a5970a4ee8/users/665702cf16a44b63baefee4149efa80b/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.201300 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13af018-1e54-473c-8533-b7d006345e2f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.210750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13af018-1e54-473c-8533-b7d006345e2f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.213940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99918634-3e6b-4a73-83a5-4f8fa3b3d11a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e34ddb26f3df4ac1980df6a5970a4ee8, user_id=665702cf16a44b63baefee4149efa80b, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:16.215292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e18242-2b8a-4c1a-bfe6-1984aae2928b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.215883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e18242-2b8a-4c1a-bfe6-1984aae2928b admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.216159 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e18242-2b8a-4c1a-bfe6-1984aae2928b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.216427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e18242-2b8a-4c1a-bfe6-1984aae2928b admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.216755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99918634-3e6b-4a73-83a5-4f8fa3b3d11a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:16.217059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e18242-2b8a-4c1a-bfe6-1984aae2928b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:16.218657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e18242-2b8a-4c1a-bfe6-1984aae2928b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:16.229784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1081/2153] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:33:16.230163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1073/2154] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:33:16 2022] PUT /identity/v3/projects/e34ddb26f3df4ac1980df6a5970a4ee8/users/665702cf16a44b63baefee4149efa80b/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:33:16.234609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e18242-2b8a-4c1a-bfe6-1984aae2928b admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.241128 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0edf44-d11c-42d0-ba05-bdf0caf855b5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.241409 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0edf44-d11c-42d0-ba05-bdf0caf855b5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.241678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0edf44-d11c-42d0-ba05-bdf0caf855b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.244166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e18242-2b8a-4c1a-bfe6-1984aae2928b admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.270726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7a0edf44-d11c-42d0-ba05-bdf0caf855b5 None None] MFA Rules not processed for user `665702cf16a44b63baefee4149efa80b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:16.271944 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7a0edf44-d11c-42d0-ba05-bdf0caf855b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:16.310916 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78025749-cb4c-4837-859b-e9e1ced8e1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.311751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78025749-cb4c-4837-859b-e9e1ced8e1b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.312166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78025749-cb4c-4837-859b-e9e1ced8e1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.312339 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78025749-cb4c-4837-859b-e9e1ced8e1b2 admin admin] PATH_INFO: `/v3/projects/a7bd597164874d2ca0c716694156cc03/users/0e16e344f43f4fb8bf89e5fa9cd4a338/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.330905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78025749-cb4c-4837-859b-e9e1ced8e1b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bd597164874d2ca0c716694156cc03, user_id=0e16e344f43f4fb8bf89e5fa9cd4a338, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:16.333699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78025749-cb4c-4837-859b-e9e1ced8e1b2 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:16.335411 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:16.350358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1082/2155] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:33:16 2022] PUT /identity/v3/projects/a7bd597164874d2ca0c716694156cc03/users/0e16e344f43f4fb8bf89e5fa9cd4a338/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:16.360824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c507057-b902-419d-92cb-13affae8590d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.361114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1074/2156] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:33:16.363039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c507057-b902-419d-92cb-13affae8590d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.363494 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c507057-b902-419d-92cb-13affae8590d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.365203 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0edf44-d11c-42d0-ba05-bdf0caf855b5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.377292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0edf44-d11c-42d0-ba05-bdf0caf855b5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.400264 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-5c507057-b902-419d-92cb-13affae8590d None None] MFA Rules not processed for user `0e16e344f43f4fb8bf89e5fa9cd4a338`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:16.401764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5c507057-b902-419d-92cb-13affae8590d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:16.452750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b4eba-fd14-47c2-8b5a-6c6aabcd370c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.453520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b4eba-fd14-47c2-8b5a-6c6aabcd370c admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.453861 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b4eba-fd14-47c2-8b5a-6c6aabcd370c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.454002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b4eba-fd14-47c2-8b5a-6c6aabcd370c admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.454891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b4eba-fd14-47c2-8b5a-6c6aabcd370c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:16.456962 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b4eba-fd14-47c2-8b5a-6c6aabcd370c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:16.476952 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1083/2157] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:33:16.488848 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c507057-b902-419d-92cb-13affae8590d None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.490427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5c507057-b902-419d-92cb-13affae8590d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:16.493676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1075/2158] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:33:16 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:33:16.494578 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b4eba-fd14-47c2-8b5a-6c6aabcd370c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.560366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c507057-b902-419d-92cb-13affae8590d None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.564038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:16.575517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b4eba-fd14-47c2-8b5a-6c6aabcd370c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.580551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6db74-cb6e-4305-b02d-dd4c9bf5d7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.581192 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6db74-cb6e-4305-b02d-dd4c9bf5d7b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.581465 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6db74-cb6e-4305-b02d-dd4c9bf5d7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.581740 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6db74-cb6e-4305-b02d-dd4c9bf5d7b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.582371 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6db74-cb6e-4305-b02d-dd4c9bf5d7b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:16.584470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6db74-cb6e-4305-b02d-dd4c9bf5d7b1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:16.605412 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1076/2159] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:33:16 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:33:16.606536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4950d-d8dd-410b-9d1d-524f40795c74 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.608374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4950d-d8dd-410b-9d1d-524f40795c74 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.608655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4950d-d8dd-410b-9d1d-524f40795c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.614521 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8b56b-4b5f-4714-b9ba-5884baab9b94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.615150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8b56b-4b5f-4714-b9ba-5884baab9b94 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.615433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8b56b-4b5f-4714-b9ba-5884baab9b94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.615706 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8b56b-4b5f-4714-b9ba-5884baab9b94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.630772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-64c4950d-d8dd-410b-9d1d-524f40795c74 None None] MFA Rules not processed for user `6d2fbc3668cf4a82b8eea0b67b87ca80`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:16.630772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-64c4950d-d8dd-410b-9d1d-524f40795c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:16.682823 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8b56b-4b5f-4714-b9ba-5884baab9b94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:16.684498 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8b56b-4b5f-4714-b9ba-5884baab9b94 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:16.694862 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1084/2160] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:33:16.695584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8b56b-4b5f-4714-b9ba-5884baab9b94 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.709366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8b56b-4b5f-4714-b9ba-5884baab9b94 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.713717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1077/2161] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:16.716824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0a736-8f13-49a5-81f1-43f9e2a8abc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.717497 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0a736-8f13-49a5-81f1-43f9e2a8abc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.717776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0a736-8f13-49a5-81f1-43f9e2a8abc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.718174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0a736-8f13-49a5-81f1-43f9e2a8abc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.718786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0a736-8f13-49a5-81f1-43f9e2a8abc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:16.720556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0a736-8f13-49a5-81f1-43f9e2a8abc4 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:16.739648 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1085/2162] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:33:16.744069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4950d-d8dd-410b-9d1d-524f40795c74 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.758640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4950d-d8dd-410b-9d1d-524f40795c74 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.764304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9039d0-8e0b-47c7-be33-7228bf897ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.765422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9039d0-8e0b-47c7-be33-7228bf897ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.765681 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9039d0-8e0b-47c7-be33-7228bf897ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.765959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9039d0-8e0b-47c7-be33-7228bf897ac2 admin admin] PATH_INFO: `/v3/projects/199def36bd194df4ab8b928b60fbc173/users/4de8b99e171647639244a2c455db2153/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.783721 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:16.802637 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9039d0-8e0b-47c7-be33-7228bf897ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=199def36bd194df4ab8b928b60fbc173, user_id=4de8b99e171647639244a2c455db2153, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:16.804994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9039d0-8e0b-47c7-be33-7228bf897ac2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:16.822009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1078/2163] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:33:16 2022] PUT /identity/v3/projects/199def36bd194df4ab8b928b60fbc173/users/4de8b99e171647639244a2c455db2153/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:16.825951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0a736-8f13-49a5-81f1-43f9e2a8abc4 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.838038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0a736-8f13-49a5-81f1-43f9e2a8abc4 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.929826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451885a5-30e0-497f-9e43-0be1f114aeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.931239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451885a5-30e0-497f-9e43-0be1f114aeaf admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.931540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451885a5-30e0-497f-9e43-0be1f114aeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.931809 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451885a5-30e0-497f-9e43-0be1f114aeaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:16.932464 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451885a5-30e0-497f-9e43-0be1f114aeaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:16.934543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451885a5-30e0-497f-9e43-0be1f114aeaf admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:16.964267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1086/2164] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:33:16.968272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9039d0-8e0b-47c7-be33-7228bf897ac2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:16.986338 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9039d0-8e0b-47c7-be33-7228bf897ac2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:16.996937 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e02284-7671-4f9d-b3fc-42fd1e86428c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:16.998137 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02284-7671-4f9d-b3fc-42fd1e86428c admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:16.998579 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02284-7671-4f9d-b3fc-42fd1e86428c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:16.999229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02284-7671-4f9d-b3fc-42fd1e86428c admin admin] PATH_INFO: `/v3/projects/199def36bd194df4ab8b928b60fbc173/users/4de8b99e171647639244a2c455db2153/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:17.001754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e02284-7671-4f9d-b3fc-42fd1e86428c admin admin] RBAC: Authorizing `identity:create_grant(project_id=199def36bd194df4ab8b928b60fbc173, user_id=4de8b99e171647639244a2c455db2153, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:17.005297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e02284-7671-4f9d-b3fc-42fd1e86428c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:17.019491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1079/2165] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:33:16 2022] PUT /identity/v3/projects/199def36bd194df4ab8b928b60fbc173/users/4de8b99e171647639244a2c455db2153/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:33:17.031621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a07a5-61dd-4a29-8412-4ffc9f12c84e None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:17.032043 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a07a5-61dd-4a29-8412-4ffc9f12c84e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:17.032436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a07a5-61dd-4a29-8412-4ffc9f12c84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:17.070631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-ba5a07a5-61dd-4a29-8412-4ffc9f12c84e None None] MFA Rules not processed for user `4de8b99e171647639244a2c455db2153`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:17.072003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ba5a07a5-61dd-4a29-8412-4ffc9f12c84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:17.138931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:17.194374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1087/2166] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:33:17.221086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e02284-7671-4f9d-b3fc-42fd1e86428c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:17.225233 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e9e02284-7671-4f9d-b3fc-42fd1e86428c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:17.304557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e02284-7671-4f9d-b3fc-42fd1e86428c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:17.308277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88600fd0-a005-4a4e-9888-07e09c2ed969 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:17.308996 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88600fd0-a005-4a4e-9888-07e09c2ed969 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:17.309391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88600fd0-a005-4a4e-9888-07e09c2ed969 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:17.309756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88600fd0-a005-4a4e-9888-07e09c2ed969 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:17.371407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88600fd0-a005-4a4e-9888-07e09c2ed969 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:17.372865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88600fd0-a005-4a4e-9888-07e09c2ed969 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:17.383388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1080/2167] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:33:17.808803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a07a5-61dd-4a29-8412-4ffc9f12c84e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:17.810221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ba5a07a5-61dd-4a29-8412-4ffc9f12c84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:17.883034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a07a5-61dd-4a29-8412-4ffc9f12c84e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:17.888013 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1124f3a-d817-49c1-b54d-90d40d782496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:17.888762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1124f3a-d817-49c1-b54d-90d40d782496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:17.889109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1124f3a-d817-49c1-b54d-90d40d782496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:17.889452 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1124f3a-d817-49c1-b54d-90d40d782496 admin admin] PATH_INFO: `/v3/users/a3b657615b2643d5829600d3afb4103c` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:17.890708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1124f3a-d817-49c1-b54d-90d40d782496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b657615b2643d5829600d3afb4103c)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:17.892706 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1124f3a-d817-49c1-b54d-90d40d782496 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:17.943499 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f1124f3a-d817-49c1-b54d-90d40d782496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b657615b2643d5829600d3afb4103c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:17.944151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f1124f3a-d817-49c1-b54d-90d40d782496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b657615b2643d5829600d3afb4103c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:17.944786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f1124f3a-d817-49c1-b54d-90d40d782496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b657615b2643d5829600d3afb4103c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:17.953462 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f1124f3a-d817-49c1-b54d-90d40d782496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b657615b2643d5829600d3afb4103c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:17.967721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f1124f3a-d817-49c1-b54d-90d40d782496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b657615b2643d5829600d3afb4103c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:17.982242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1088/2168] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:33:17 2022] DELETE /identity/v3/users/a3b657615b2643d5829600d3afb4103c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:17.986788 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5eb91-3fa8-495b-81e7-ef6cc05746ca None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:17.987274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5eb91-3fa8-495b-81e7-ef6cc05746ca None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:17.987423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5eb91-3fa8-495b-81e7-ef6cc05746ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:18.008028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-24a5eb91-3fa8-495b-81e7-ef6cc05746ca None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:18.009391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-24a5eb91-3fa8-495b-81e7-ef6cc05746ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:18.070321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1081/2169] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:18.081127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1124f3a-d817-49c1-b54d-90d40d782496 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:18.083282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f1124f3a-d817-49c1-b54d-90d40d782496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:18.143600 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1124f3a-d817-49c1-b54d-90d40d782496 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:18.148093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56041f0a-fbdb-488a-a4d5-c03c11c93608 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:18.148093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56041f0a-fbdb-488a-a4d5-c03c11c93608 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:18.148386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56041f0a-fbdb-488a-a4d5-c03c11c93608 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:18.148641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56041f0a-fbdb-488a-a4d5-c03c11c93608 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:18.154878 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:18.160052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56041f0a-fbdb-488a-a4d5-c03c11c93608 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:18.161967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56041f0a-fbdb-488a-a4d5-c03c11c93608 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:18.172799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1089/2170] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:33:18.276639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef417ef7-3dda-4d31-bfd0-1ca944cda7f6 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:18.277006 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef417ef7-3dda-4d31-bfd0-1ca944cda7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:18.277301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef417ef7-3dda-4d31-bfd0-1ca944cda7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:18.315531 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ef417ef7-3dda-4d31-bfd0-1ca944cda7f6 None None] MFA Rules not processed for user `0e16e344f43f4fb8bf89e5fa9cd4a338`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:18.317009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ef417ef7-3dda-4d31-bfd0-1ca944cda7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:18.340311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:18.433275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1082/2171] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:18.581637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56041f0a-fbdb-488a-a4d5-c03c11c93608 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:18.584946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-56041f0a-fbdb-488a-a4d5-c03c11c93608 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:18.614991 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56041f0a-fbdb-488a-a4d5-c03c11c93608 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:18.621411 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:18.622062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:18.622336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:18.622643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] PATH_INFO: `/v3/projects/89efe72a8514421d84221b296f6b7f34` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:18.623778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89efe72a8514421d84221b296f6b7f34)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:18.625699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:18.674757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89efe72a8514421d84221b296f6b7f34', 'request_id': 'req-32940d8f-5492-4344-a3e7-15a48687f0c5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:18.681046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89efe72a8514421d84221b296f6b7f34', 'request_id': 'req-32940d8f-5492-4344-a3e7-15a48687f0c5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:18.689104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89efe72a8514421d84221b296f6b7f34', 'request_id': 'req-32940d8f-5492-4344-a3e7-15a48687f0c5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:18.700275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] The token cache is being invalidate because project 89efe72a8514421d84221b296f6b7f34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:33:18.700562 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:18.701185 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:18.702726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1090/2172] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:33:18 2022] DELETE /identity/v3/projects/89efe72a8514421d84221b296f6b7f34 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:22.281095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef417ef7-3dda-4d31-bfd0-1ca944cda7f6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:22.283118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ef417ef7-3dda-4d31-bfd0-1ca944cda7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:22.348469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef417ef7-3dda-4d31-bfd0-1ca944cda7f6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:22.410325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51febba6-8b8f-4a87-aa00-890a64660846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:22.411923 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51febba6-8b8f-4a87-aa00-890a64660846 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:22.412376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51febba6-8b8f-4a87-aa00-890a64660846 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:22.412807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51febba6-8b8f-4a87-aa00-890a64660846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:22.415191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-51febba6-8b8f-4a87-aa00-890a64660846 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:22.503945 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51febba6-8b8f-4a87-aa00-890a64660846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:22.506863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51febba6-8b8f-4a87-aa00-890a64660846 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:22.518654 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1083/2173] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:33:24.410958 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:24.413083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:24.519823 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32940d8f-5492-4344-a3e7-15a48687f0c5 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:24.526277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5bae5-add1-4127-a440-3a22f108a92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:24.527216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5bae5-add1-4127-a440-3a22f108a92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:24.527595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5bae5-add1-4127-a440-3a22f108a92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:24.528107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5bae5-add1-4127-a440-3a22f108a92c admin admin] PATH_INFO: `/v3/users/1ce1cd9c19584549b794df8e487abf00` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:24.529782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5bae5-add1-4127-a440-3a22f108a92c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ce1cd9c19584549b794df8e487abf00)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:24.532245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5bae5-add1-4127-a440-3a22f108a92c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:24.603320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-6dd5bae5-add1-4127-a440-3a22f108a92c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ce1cd9c19584549b794df8e487abf00'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:24.604126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-6dd5bae5-add1-4127-a440-3a22f108a92c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ce1cd9c19584549b794df8e487abf00'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:24.604794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-6dd5bae5-add1-4127-a440-3a22f108a92c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ce1cd9c19584549b794df8e487abf00'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:24.613716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-6dd5bae5-add1-4127-a440-3a22f108a92c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ce1cd9c19584549b794df8e487abf00'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:24.627545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-6dd5bae5-add1-4127-a440-3a22f108a92c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ce1cd9c19584549b794df8e487abf00'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:24.634438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:24.653949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1091/2174] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:33:24 2022] DELETE /identity/v3/users/1ce1cd9c19584549b794df8e487abf00 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:24.854627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51febba6-8b8f-4a87-aa00-890a64660846 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:24.869086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51febba6-8b8f-4a87-aa00-890a64660846 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:24.906453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:24.932439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:24.933061 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:24.933341 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:24.933608 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2 admin admin] PATH_INFO: `/v3/projects/1f227ee89ce84dec84410a1db9b479fc` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:24.934823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f227ee89ce84dec84410a1db9b479fc)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:24.936726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:24.986889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f227ee89ce84dec84410a1db9b479fc', 'request_id': 'req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:24.993022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f227ee89ce84dec84410a1db9b479fc', 'request_id': 'req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:24.998574 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f227ee89ce84dec84410a1db9b479fc', 'request_id': 'req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:25.010799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2 admin admin] The token cache is being invalidate because project 1f227ee89ce84dec84410a1db9b479fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:33:25.011214 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:25.011831 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2771f5da-013e-441b-9bc6-6dcd7dd6abe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:25.013306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1084/2175] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:33:24 2022] DELETE /identity/v3/projects/1f227ee89ce84dec84410a1db9b479fc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:33:25.018615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fba1e-445a-454d-b70f-b38f47e4ba8d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:25.018973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fba1e-445a-454d-b70f-b38f47e4ba8d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:25.019256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fba1e-445a-454d-b70f-b38f47e4ba8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:25.039921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-1e3fba1e-445a-454d-b70f-b38f47e4ba8d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:25.041083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1e3fba1e-445a-454d-b70f-b38f47e4ba8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:25.104244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1092/2176] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:25.112325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c163c63-422a-4e2f-973d-e87ce0bec0a9 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:25.112634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c163c63-422a-4e2f-973d-e87ce0bec0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:25.112911 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c163c63-422a-4e2f-973d-e87ce0bec0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:25.133974 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1c163c63-422a-4e2f-973d-e87ce0bec0a9 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:25.135222 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1c163c63-422a-4e2f-973d-e87ce0bec0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:25.144011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1085/2177] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:25.148485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fba1e-445a-454d-b70f-b38f47e4ba8d None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:25.158195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fba1e-445a-454d-b70f-b38f47e4ba8d None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:25.163688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab218b-e650-4ad5-b491-08ae8049aaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:25.164290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab218b-e650-4ad5-b491-08ae8049aaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:25.164567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab218b-e650-4ad5-b491-08ae8049aaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:25.164838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab218b-e650-4ad5-b491-08ae8049aaa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:25.165492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab218b-e650-4ad5-b491-08ae8049aaa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:33:25.165792 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab218b-e650-4ad5-b491-08ae8049aaa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:25.167403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab218b-e650-4ad5-b491-08ae8049aaa6 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:25.179245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1093/2178] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:33:25.183521 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c163c63-422a-4e2f-973d-e87ce0bec0a9 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:25.192927 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c163c63-422a-4e2f-973d-e87ce0bec0a9 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:25.197866 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8c0a3-1302-4bc7-84da-f2c8e9c532fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:25.198453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8c0a3-1302-4bc7-84da-f2c8e9c532fb admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:25.198827 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8c0a3-1302-4bc7-84da-f2c8e9c532fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:25.199103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8c0a3-1302-4bc7-84da-f2c8e9c532fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:25.199721 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8c0a3-1302-4bc7-84da-f2c8e9c532fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:25.201616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8c0a3-1302-4bc7-84da-f2c8e9c532fb admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:25.230417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1086/2179] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:33:25 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:33:25.234771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab218b-e650-4ad5-b491-08ae8049aaa6 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:25.244883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab218b-e650-4ad5-b491-08ae8049aaa6 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:25.311437 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e34d4-c6a5-4f01-b1cf-b311acca3949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:25.311953 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e34d4-c6a5-4f01-b1cf-b311acca3949 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:25.312237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e34d4-c6a5-4f01-b1cf-b311acca3949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:25.312501 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e34d4-c6a5-4f01-b1cf-b311acca3949 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:25.313118 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e34d4-c6a5-4f01-b1cf-b311acca3949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:25.315114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e34d4-c6a5-4f01-b1cf-b311acca3949 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:25.337726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1094/2180] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:33:25 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:33:25.342088 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8c0a3-1302-4bc7-84da-f2c8e9c532fb admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:25.352960 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8c0a3-1302-4bc7-84da-f2c8e9c532fb admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:25.359001 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122061fd-ba41-42c1-9e83-13e3e290504c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:25.359619 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122061fd-ba41-42c1-9e83-13e3e290504c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:25.359890 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122061fd-ba41-42c1-9e83-13e3e290504c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:25.360159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122061fd-ba41-42c1-9e83-13e3e290504c admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:25.360789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122061fd-ba41-42c1-9e83-13e3e290504c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:25.362366 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122061fd-ba41-42c1-9e83-13e3e290504c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:25.372768 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1087/2181] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:33:25.376694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e34d4-c6a5-4f01-b1cf-b311acca3949 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:25.387270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e34d4-c6a5-4f01-b1cf-b311acca3949 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:25.392538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9be19a-d934-459e-af3e-e97d9f265d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:25.393129 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9be19a-d934-459e-af3e-e97d9f265d8d admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:25.393405 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9be19a-d934-459e-af3e-e97d9f265d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:25.393671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9be19a-d934-459e-af3e-e97d9f265d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:25.394660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9be19a-d934-459e-af3e-e97d9f265d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:25.396324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9be19a-d934-459e-af3e-e97d9f265d8d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:25.407209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1095/2182] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:33:25.411643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122061fd-ba41-42c1-9e83-13e3e290504c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:25.422747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122061fd-ba41-42c1-9e83-13e3e290504c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:25.427830 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f66f15-77f6-41e9-913c-6801c77397a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:25.428740 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f66f15-77f6-41e9-913c-6801c77397a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:25.429119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f66f15-77f6-41e9-913c-6801c77397a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:25.429399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f66f15-77f6-41e9-913c-6801c77397a3 admin admin] PATH_INFO: `/v3/projects/3fca9208950f4d1a92bd52b4e5f5b2a1/users/f8bd900e231242c3ae67432d26d089da/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:25.449501 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f66f15-77f6-41e9-913c-6801c77397a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fca9208950f4d1a92bd52b4e5f5b2a1, user_id=f8bd900e231242c3ae67432d26d089da, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:25.452231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f66f15-77f6-41e9-913c-6801c77397a3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:25.466063 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1088/2183] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:33:25 2022] PUT /identity/v3/projects/3fca9208950f4d1a92bd52b4e5f5b2a1/users/f8bd900e231242c3ae67432d26d089da/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:25.471721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c8a84-69fe-4e8a-a38f-4eaac0a1f7e0 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:25.472239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c8a84-69fe-4e8a-a38f-4eaac0a1f7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:25.472482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c8a84-69fe-4e8a-a38f-4eaac0a1f7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:25.492387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-af1c8a84-69fe-4e8a-a38f-4eaac0a1f7e0 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:25.493603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-af1c8a84-69fe-4e8a-a38f-4eaac0a1f7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:25.562642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1096/2184] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:25.571651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f66f15-77f6-41e9-913c-6801c77397a3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:25.574049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-98f66f15-77f6-41e9-913c-6801c77397a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:25.638440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f66f15-77f6-41e9-913c-6801c77397a3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:25.642609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21b2dd-3ed5-442c-b432-443a44ffbe22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:25.643215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21b2dd-3ed5-442c-b432-443a44ffbe22 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:25.643633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21b2dd-3ed5-442c-b432-443a44ffbe22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:25.643793 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21b2dd-3ed5-442c-b432-443a44ffbe22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:25.655467 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21b2dd-3ed5-442c-b432-443a44ffbe22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:25.659427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21b2dd-3ed5-442c-b432-443a44ffbe22 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:25.670290 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1089/2185] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:33:27.932954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c8a84-69fe-4e8a-a38f-4eaac0a1f7e0 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:27.934602 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-af1c8a84-69fe-4e8a-a38f-4eaac0a1f7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:27.943552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:27.950452 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c8a84-69fe-4e8a-a38f-4eaac0a1f7e0 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:27.960726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4179995-0a66-49ef-8697-3eb3daa6c1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:27.961567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4179995-0a66-49ef-8697-3eb3daa6c1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:27.961992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4179995-0a66-49ef-8697-3eb3daa6c1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:27.962412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4179995-0a66-49ef-8697-3eb3daa6c1af admin admin] PATH_INFO: `/v3/users/665702cf16a44b63baefee4149efa80b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:27.963876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4179995-0a66-49ef-8697-3eb3daa6c1af admin admin] RBAC: Authorizing `identity:delete_user(user_id=665702cf16a44b63baefee4149efa80b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:27.966161 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4179995-0a66-49ef-8697-3eb3daa6c1af admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:28.053028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c4179995-0a66-49ef-8697-3eb3daa6c1af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '665702cf16a44b63baefee4149efa80b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:28.053991 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c4179995-0a66-49ef-8697-3eb3daa6c1af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '665702cf16a44b63baefee4149efa80b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:28.054791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c4179995-0a66-49ef-8697-3eb3daa6c1af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '665702cf16a44b63baefee4149efa80b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:28.065094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c4179995-0a66-49ef-8697-3eb3daa6c1af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '665702cf16a44b63baefee4149efa80b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:28.100274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c4179995-0a66-49ef-8697-3eb3daa6c1af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '665702cf16a44b63baefee4149efa80b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:28.122843 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1097/2186] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:33:27 2022] DELETE /identity/v3/users/665702cf16a44b63baefee4149efa80b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:28.124614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2aa35-6711-4cb3-be53-a629da0a35b8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:28.125082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2aa35-6711-4cb3-be53-a629da0a35b8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:28.125190 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2aa35-6711-4cb3-be53-a629da0a35b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:28.137527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:28.164021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-91b2aa35-6711-4cb3-be53-a629da0a35b8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:28.165676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-91b2aa35-6711-4cb3-be53-a629da0a35b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:28.293360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1090/2187] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:33:28.303881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4179995-0a66-49ef-8697-3eb3daa6c1af admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:28.305616 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c4179995-0a66-49ef-8697-3eb3daa6c1af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:28.449469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4179995-0a66-49ef-8697-3eb3daa6c1af admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:28.456677 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2c22a-1e3e-49ec-bd9a-4dca3fb47356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:28.457631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2c22a-1e3e-49ec-bd9a-4dca3fb47356 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:28.458454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2c22a-1e3e-49ec-bd9a-4dca3fb47356 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:28.459048 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2c22a-1e3e-49ec-bd9a-4dca3fb47356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:28.472419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2c22a-1e3e-49ec-bd9a-4dca3fb47356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:28.475494 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2c22a-1e3e-49ec-bd9a-4dca3fb47356 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:28.499332 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1098/2188] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:33:29.140819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2aa35-6711-4cb3-be53-a629da0a35b8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:29.142981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-91b2aa35-6711-4cb3-be53-a629da0a35b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:29.157418 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2aa35-6711-4cb3-be53-a629da0a35b8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:29.162191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:29.162972 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:29.163456 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:29.163940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] PATH_INFO: `/v3/projects/e34ddb26f3df4ac1980df6a5970a4ee8` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:29.165282 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e34ddb26f3df4ac1980df6a5970a4ee8)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:29.167558 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:29.230902 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e34ddb26f3df4ac1980df6a5970a4ee8', 'request_id': 'req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:29.247416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e34ddb26f3df4ac1980df6a5970a4ee8', 'request_id': 'req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:29.256233 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e34ddb26f3df4ac1980df6a5970a4ee8', 'request_id': 'req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:29.284381 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] The token cache is being invalidate because project e34ddb26f3df4ac1980df6a5970a4ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:33:29.285024 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:29.286244 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:29.293006 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1091/2189] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:33:29 2022] DELETE /identity/v3/projects/e34ddb26f3df4ac1980df6a5970a4ee8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:29.294448 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2c22a-1e3e-49ec-bd9a-4dca3fb47356 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:29.316802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2c22a-1e3e-49ec-bd9a-4dca3fb47356 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:29.448068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759260d-f0f7-4496-9bde-d84813664aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:29.449564 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759260d-f0f7-4496-9bde-d84813664aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:29.449961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759260d-f0f7-4496-9bde-d84813664aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:29.450377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759260d-f0f7-4496-9bde-d84813664aea admin admin] PATH_INFO: `/v3/users/4de8b99e171647639244a2c455db2153` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:29.452114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759260d-f0f7-4496-9bde-d84813664aea admin admin] RBAC: Authorizing `identity:delete_user(user_id=4de8b99e171647639244a2c455db2153)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:29.454667 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759260d-f0f7-4496-9bde-d84813664aea admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:29.490167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:29.560705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0759260d-f0f7-4496-9bde-d84813664aea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4de8b99e171647639244a2c455db2153'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:29.561517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0759260d-f0f7-4496-9bde-d84813664aea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4de8b99e171647639244a2c455db2153'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:29.562220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0759260d-f0f7-4496-9bde-d84813664aea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4de8b99e171647639244a2c455db2153'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:29.583332 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0759260d-f0f7-4496-9bde-d84813664aea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4de8b99e171647639244a2c455db2153'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:29.609286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0759260d-f0f7-4496-9bde-d84813664aea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4de8b99e171647639244a2c455db2153'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:29.644395 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1099/2190] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:33:29 2022] DELETE /identity/v3/users/4de8b99e171647639244a2c455db2153 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:30.237331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:30.239274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:30.256457 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10bb2e-a5c3-4c8a-a7bb-9347a7c15e51 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:30.322034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:30.322717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:30.322997 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:30.323267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] PATH_INFO: `/v3/projects/199def36bd194df4ab8b928b60fbc173` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:30.324430 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=199def36bd194df4ab8b928b60fbc173)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:30.326353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:30.367002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:30.391240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '199def36bd194df4ab8b928b60fbc173', 'request_id': 'req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:30.401396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '199def36bd194df4ab8b928b60fbc173', 'request_id': 'req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:30.413226 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '199def36bd194df4ab8b928b60fbc173', 'request_id': 'req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:30.435138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] The token cache is being invalidate because project 199def36bd194df4ab8b928b60fbc173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:33:30.435430 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:30.436124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:30.438008 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1092/2191] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:33:30 2022] DELETE /identity/v3/projects/199def36bd194df4ab8b928b60fbc173 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:30.678272 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41f313-57f2-4093-b2a8-4958a93a9e2f None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:30.678788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41f313-57f2-4093-b2a8-4958a93a9e2f None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:30.678878 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41f313-57f2-4093-b2a8-4958a93a9e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:30.716529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-ec41f313-57f2-4093-b2a8-4958a93a9e2f None None] MFA Rules not processed for user `f8bd900e231242c3ae67432d26d089da`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:30.717773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ec41f313-57f2-4093-b2a8-4958a93a9e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:30.805626 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1100/2192] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:33:30.819957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:30.821370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:30.914607 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4c3fd-c7fb-42d4-835b-4fc6d38bf508 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:30.920311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b17139-3307-4f97-bf4e-da742e05e1ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:30.920944 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17139-3307-4f97-bf4e-da742e05e1ed service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:30.921223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17139-3307-4f97-bf4e-da742e05e1ed service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:30.921495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17139-3307-4f97-bf4e-da742e05e1ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:30.930752 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b17139-3307-4f97-bf4e-da742e05e1ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:30.932368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b17139-3307-4f97-bf4e-da742e05e1ed service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:30.941880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1093/2193] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:33:33.658620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446502b8-3ffd-4237-8f46-0f2b1470684e None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:33.659087 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446502b8-3ffd-4237-8f46-0f2b1470684e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:33.659176 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446502b8-3ffd-4237-8f46-0f2b1470684e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:33.678704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-446502b8-3ffd-4237-8f46-0f2b1470684e None None] MFA Rules not processed for user `f8bd900e231242c3ae67432d26d089da`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:33.679918 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-446502b8-3ffd-4237-8f46-0f2b1470684e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:33.686768 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1101/2194] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:40.784877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be524c-541c-4bff-a0eb-62f59e520de2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:40.785361 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be524c-541c-4bff-a0eb-62f59e520de2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:40.785558 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be524c-541c-4bff-a0eb-62f59e520de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:40.806810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c8be524c-541c-4bff-a0eb-62f59e520de2 None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:40.808042 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c8be524c-541c-4bff-a0eb-62f59e520de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:40.872708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1094/2195] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:33:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:40.920296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446502b8-3ffd-4237-8f46-0f2b1470684e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:40.922197 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-446502b8-3ffd-4237-8f46-0f2b1470684e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:41.008822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446502b8-3ffd-4237-8f46-0f2b1470684e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:41.016156 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276203e-bab6-4d1d-b71d-c2705443b007 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:41.017008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276203e-bab6-4d1d-b71d-c2705443b007 service swift] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:41.017422 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276203e-bab6-4d1d-b71d-c2705443b007 service swift] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:41.017828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276203e-bab6-4d1d-b71d-c2705443b007 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:41.027166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276203e-bab6-4d1d-b71d-c2705443b007 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:41.028976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276203e-bab6-4d1d-b71d-c2705443b007 service swift] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:41.038783 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1102/2196] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:33:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:33:41.114694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15582-fb14-489b-8f6c-fe0b2fbda98d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:41.115592 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15582-fb14-489b-8f6c-fe0b2fbda98d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:41.115949 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15582-fb14-489b-8f6c-fe0b2fbda98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:41.137379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3ce15582-fb14-489b-8f6c-fe0b2fbda98d None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:41.140839 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3ce15582-fb14-489b-8f6c-fe0b2fbda98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:41.146433 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1095/2197] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:33:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:41.162580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276203e-bab6-4d1d-b71d-c2705443b007 service swift] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:41.174643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276203e-bab6-4d1d-b71d-c2705443b007 service swift] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:41.182753 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e64369-b757-418c-81d2-b5401d99b632 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:41.183699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e64369-b757-418c-81d2-b5401d99b632 service swift] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:41.183975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e64369-b757-418c-81d2-b5401d99b632 service swift] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:41.184242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e64369-b757-418c-81d2-b5401d99b632 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:41.192282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:41.196799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e64369-b757-418c-81d2-b5401d99b632 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:41.199368 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e64369-b757-418c-81d2-b5401d99b632 service swift] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:41.209423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1103/2198] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:33:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:33:41.685319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683b782-d3d7-4eda-8f01-41a3823dc841 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:41.687167 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683b782-d3d7-4eda-8f01-41a3823dc841 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:41.687543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683b782-d3d7-4eda-8f01-41a3823dc841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:41.714676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3683b782-d3d7-4eda-8f01-41a3823dc841 None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:41.716453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3683b782-d3d7-4eda-8f01-41a3823dc841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:41.727453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1096/2199] 10.222.0.23 () {58 vars in 967 bytes} [Fri Sep 9 16:33:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:41.748344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e64369-b757-418c-81d2-b5401d99b632 service swift] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:41.771931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e64369-b757-418c-81d2-b5401d99b632 service swift] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:41.787996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db359a-29da-492c-8ff5-5b5927545707 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:41.789153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db359a-29da-492c-8ff5-5b5927545707 service swift] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:41.789689 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db359a-29da-492c-8ff5-5b5927545707 service swift] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:41.790245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db359a-29da-492c-8ff5-5b5927545707 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:41.809311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db359a-29da-492c-8ff5-5b5927545707 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:41.811090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db359a-29da-492c-8ff5-5b5927545707 service swift] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:41.820712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1104/2200] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:33:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:33:42.407295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6be6e-4cca-4668-8925-9f14e40449a1 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:42.407823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6be6e-4cca-4668-8925-9f14e40449a1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:42.407915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6be6e-4cca-4668-8925-9f14e40449a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:42.439821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-2db6be6e-4cca-4668-8925-9f14e40449a1 None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:42.441112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2db6be6e-4cca-4668-8925-9f14e40449a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:42.458311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1097/2201] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:33:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:33:42.465375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db359a-29da-492c-8ff5-5b5927545707 service swift] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:42.477019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db359a-29da-492c-8ff5-5b5927545707 service swift] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:42.483175 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6d975-71d7-4ded-ae13-ad530c4a019c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:42.484298 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6d975-71d7-4ded-ae13-ad530c4a019c service swift] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:42.484822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6d975-71d7-4ded-ae13-ad530c4a019c service swift] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:42.485288 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6d975-71d7-4ded-ae13-ad530c4a019c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:42.497408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc6d975-71d7-4ded-ae13-ad530c4a019c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:42.499330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc6d975-71d7-4ded-ae13-ad530c4a019c service swift] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:42.529366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1105/2202] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:33:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 16:33:43.157966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c920b-ecd6-4fdd-ae90-0d73e40a7fe0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:43.158255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c920b-ecd6-4fdd-ae90-0d73e40a7fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:43.158543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c920b-ecd6-4fdd-ae90-0d73e40a7fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:43.179166 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-885c920b-ecd6-4fdd-ae90-0d73e40a7fe0 None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:43.180379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-885c920b-ecd6-4fdd-ae90-0d73e40a7fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:43.187899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1098/2203] 10.222.0.23 () {58 vars in 967 bytes} [Fri Sep 9 16:33:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:33:43.202697 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6d975-71d7-4ded-ae13-ad530c4a019c service swift] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:43.213693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6d975-71d7-4ded-ae13-ad530c4a019c service swift] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:43.219001 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb5a02-2381-4f0a-bf7b-f3e4a184804d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:43.219620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb5a02-2381-4f0a-bf7b-f3e4a184804d service swift] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:43.219902 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb5a02-2381-4f0a-bf7b-f3e4a184804d service swift] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:43.220174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb5a02-2381-4f0a-bf7b-f3e4a184804d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:43.229495 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb5a02-2381-4f0a-bf7b-f3e4a184804d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:43.231142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb5a02-2381-4f0a-bf7b-f3e4a184804d service swift] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:43.240547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1106/2204] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:33:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 16:33:46.241294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c920b-ecd6-4fdd-ae90-0d73e40a7fe0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:46.244859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-885c920b-ecd6-4fdd-ae90-0d73e40a7fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:46.294270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:46.346893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c920b-ecd6-4fdd-ae90-0d73e40a7fe0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:46.351840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5a7cb-4209-404b-9e35-d772717a96ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:46.353454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5a7cb-4209-404b-9e35-d772717a96ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:46.353872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5a7cb-4209-404b-9e35-d772717a96ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:46.354147 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5a7cb-4209-404b-9e35-d772717a96ed admin admin] PATH_INFO: `/v3/users/0e16e344f43f4fb8bf89e5fa9cd4a338` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:46.355880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5a7cb-4209-404b-9e35-d772717a96ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e16e344f43f4fb8bf89e5fa9cd4a338)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:46.358325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5a7cb-4209-404b-9e35-d772717a96ed admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:46.456546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77d5a7cb-4209-404b-9e35-d772717a96ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e16e344f43f4fb8bf89e5fa9cd4a338'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:46.457202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77d5a7cb-4209-404b-9e35-d772717a96ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e16e344f43f4fb8bf89e5fa9cd4a338'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:46.458396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77d5a7cb-4209-404b-9e35-d772717a96ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e16e344f43f4fb8bf89e5fa9cd4a338'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:46.470993 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77d5a7cb-4209-404b-9e35-d772717a96ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e16e344f43f4fb8bf89e5fa9cd4a338'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:46.504891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77d5a7cb-4209-404b-9e35-d772717a96ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e16e344f43f4fb8bf89e5fa9cd4a338'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:46.537087 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1099/2205] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:33:46 2022] DELETE /identity/v3/users/0e16e344f43f4fb8bf89e5fa9cd4a338 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:46.538433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc155e9-447a-40da-bae4-e0de0ccb46c8 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:46.538959 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc155e9-447a-40da-bae4-e0de0ccb46c8 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:46.539050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc155e9-447a-40da-bae4-e0de0ccb46c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:46.574169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-7fc155e9-447a-40da-bae4-e0de0ccb46c8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:46.575415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7fc155e9-447a-40da-bae4-e0de0ccb46c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:46.648914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1107/2206] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:33:46.658744 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5a7cb-4209-404b-9e35-d772717a96ed admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:46.660608 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-77d5a7cb-4209-404b-9e35-d772717a96ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:46.733983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5a7cb-4209-404b-9e35-d772717a96ed admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:46.739012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f293a8-34c1-4a9d-9ecc-9feb27255057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:46.739819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f293a8-34c1-4a9d-9ecc-9feb27255057 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:46.740288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f293a8-34c1-4a9d-9ecc-9feb27255057 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:46.740656 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f293a8-34c1-4a9d-9ecc-9feb27255057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:46.753684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f293a8-34c1-4a9d-9ecc-9feb27255057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:46.755210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f293a8-34c1-4a9d-9ecc-9feb27255057 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:46.765282 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1100/2207] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:33:47.035747 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc155e9-447a-40da-bae4-e0de0ccb46c8 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:47.037876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7fc155e9-447a-40da-bae4-e0de0ccb46c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:47.057748 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc155e9-447a-40da-bae4-e0de0ccb46c8 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:47.075941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:47.077027 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:47.077489 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:47.077852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2 admin admin] PATH_INFO: `/v3/projects/a7bd597164874d2ca0c716694156cc03` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:47.079663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7bd597164874d2ca0c716694156cc03)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:47.082707 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:47.155398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7bd597164874d2ca0c716694156cc03', 'request_id': 'req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:47.162334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7bd597164874d2ca0c716694156cc03', 'request_id': 'req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:47.173260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7bd597164874d2ca0c716694156cc03', 'request_id': 'req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:47.184657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2 admin admin] The token cache is being invalidate because project a7bd597164874d2ca0c716694156cc03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:33:47.184989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:47.185623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0e4269a6-470e-4fa0-9c08-b73c5f1dfca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:33:47.188490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1108/2208] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:33:47 2022] DELETE /identity/v3/projects/a7bd597164874d2ca0c716694156cc03 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:33:54.119321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17d20f-2db6-48b9-a429-3a812a6d05eb None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:54.119743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17d20f-2db6-48b9-a429-3a812a6d05eb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:54.119926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17d20f-2db6-48b9-a429-3a812a6d05eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:54.143523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ae17d20f-2db6-48b9-a429-3a812a6d05eb None None] MFA Rules not processed for user `53c0721dc18343b18993ff5af098841d`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:54.144739 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ae17d20f-2db6-48b9-a429-3a812a6d05eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:54.195136 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:54.215207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1101/2209] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:33:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:57.181609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:33:59.551873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b4f64-fec0-43db-b6fe-3a634fe622fb None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:59.552492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b4f64-fec0-43db-b6fe-3a634fe622fb None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:59.552492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b4f64-fec0-43db-b6fe-3a634fe622fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:59.574003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-902b4f64-fec0-43db-b6fe-3a634fe622fb None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:33:59.575245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-902b4f64-fec0-43db-b6fe-3a634fe622fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:59.638485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1109/2210] 10.222.0.23 () {58 vars in 967 bytes} [Fri Sep 9 16:33:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:33:59.655374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17d20f-2db6-48b9-a429-3a812a6d05eb None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:33:59.657049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ae17d20f-2db6-48b9-a429-3a812a6d05eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:33:59.741360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17d20f-2db6-48b9-a429-3a812a6d05eb None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:33:59.747481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93740d55-9e95-4f72-aedb-92c1399044f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:33:59.748082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93740d55-9e95-4f72-aedb-92c1399044f6 service swift] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:33:59.748377 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93740d55-9e95-4f72-aedb-92c1399044f6 service swift] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:33:59.748615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93740d55-9e95-4f72-aedb-92c1399044f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:33:59.757161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93740d55-9e95-4f72-aedb-92c1399044f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:33:59.758854 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93740d55-9e95-4f72-aedb-92c1399044f6 service swift] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:33:59.766919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1102/2211] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:33:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:34:01.897697 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b4f64-fec0-43db-b6fe-3a634fe622fb None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:01.899187 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-902b4f64-fec0-43db-b6fe-3a634fe622fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:01.965816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b4f64-fec0-43db-b6fe-3a634fe622fb None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:01.970825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a0353-261a-41fd-b346-1f2c61fe8bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:01.971079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a0353-261a-41fd-b346-1f2c61fe8bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:01.971356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a0353-261a-41fd-b346-1f2c61fe8bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:01.971625 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a0353-261a-41fd-b346-1f2c61fe8bf6 admin admin] PATH_INFO: `/v3/users/6d2fbc3668cf4a82b8eea0b67b87ca80` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:01.972786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a0353-261a-41fd-b346-1f2c61fe8bf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d2fbc3668cf4a82b8eea0b67b87ca80)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:01.974819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a0353-261a-41fd-b346-1f2c61fe8bf6 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:02.026573 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2d1a0353-261a-41fd-b346-1f2c61fe8bf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d2fbc3668cf4a82b8eea0b67b87ca80'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:02.027238 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2d1a0353-261a-41fd-b346-1f2c61fe8bf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d2fbc3668cf4a82b8eea0b67b87ca80'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:02.027827 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2d1a0353-261a-41fd-b346-1f2c61fe8bf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d2fbc3668cf4a82b8eea0b67b87ca80'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:02.035049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2d1a0353-261a-41fd-b346-1f2c61fe8bf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d2fbc3668cf4a82b8eea0b67b87ca80'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:02.047459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2d1a0353-261a-41fd-b346-1f2c61fe8bf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d2fbc3668cf4a82b8eea0b67b87ca80'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:02.061300 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1110/2212] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:34:01 2022] DELETE /identity/v3/users/6d2fbc3668cf4a82b8eea0b67b87ca80 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:34:02.290775 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93740d55-9e95-4f72-aedb-92c1399044f6 service swift] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:02.304498 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93740d55-9e95-4f72-aedb-92c1399044f6 service swift] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:02.390815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc15f2-169f-4806-969a-397f705db1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:02.391851 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc15f2-169f-4806-969a-397f705db1cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:02.392322 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc15f2-169f-4806-969a-397f705db1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:02.392802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc15f2-169f-4806-969a-397f705db1cc admin admin] PATH_INFO: `/v3/projects/e9243cdc25d5444d847db4f59c05fa6a` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:02.394676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc15f2-169f-4806-969a-397f705db1cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9243cdc25d5444d847db4f59c05fa6a)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:02.396871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc15f2-169f-4806-969a-397f705db1cc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:02.470196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a1cc15f2-169f-4806-969a-397f705db1cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9243cdc25d5444d847db4f59c05fa6a', 'request_id': 'req-a1cc15f2-169f-4806-969a-397f705db1cc'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:02.488297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a1cc15f2-169f-4806-969a-397f705db1cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9243cdc25d5444d847db4f59c05fa6a', 'request_id': 'req-a1cc15f2-169f-4806-969a-397f705db1cc'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:02.495097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a1cc15f2-169f-4806-969a-397f705db1cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9243cdc25d5444d847db4f59c05fa6a', 'request_id': 'req-a1cc15f2-169f-4806-969a-397f705db1cc'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:02.508247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a1cc15f2-169f-4806-969a-397f705db1cc admin admin] The token cache is being invalidate because project e9243cdc25d5444d847db4f59c05fa6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:34:02.508650 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a1cc15f2-169f-4806-969a-397f705db1cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:02.509335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a1cc15f2-169f-4806-969a-397f705db1cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:02.510877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1103/2213] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:34:02 2022] DELETE /identity/v3/projects/e9243cdc25d5444d847db4f59c05fa6a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:34:02.516235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad3701-b02c-4c33-b2b8-d6e29c7fa20b None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:02.516796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad3701-b02c-4c33-b2b8-d6e29c7fa20b None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:02.517138 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad3701-b02c-4c33-b2b8-d6e29c7fa20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:02.538317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-15ad3701-b02c-4c33-b2b8-d6e29c7fa20b None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:34:02.539491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-15ad3701-b02c-4c33-b2b8-d6e29c7fa20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:02.558619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:34:02.603569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1111/2214] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:34:02.611355 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab395f-225b-4b22-9908-053ba7af7671 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:02.611575 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab395f-225b-4b22-9908-053ba7af7671 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:02.611842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab395f-225b-4b22-9908-053ba7af7671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:02.635743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-16ab395f-225b-4b22-9908-053ba7af7671 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:34:02.637652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-16ab395f-225b-4b22-9908-053ba7af7671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:02.645856 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1104/2215] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:34:02.650195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad3701-b02c-4c33-b2b8-d6e29c7fa20b None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:02.660286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad3701-b02c-4c33-b2b8-d6e29c7fa20b None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:02.665269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d91f9-f1f4-436a-96e5-098e13a0ad3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:02.670705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d91f9-f1f4-436a-96e5-098e13a0ad3d admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:02.670705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d91f9-f1f4-436a-96e5-098e13a0ad3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:02.670705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d91f9-f1f4-436a-96e5-098e13a0ad3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:02.670705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d91f9-f1f4-436a-96e5-098e13a0ad3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:34:02.670705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d91f9-f1f4-436a-96e5-098e13a0ad3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:02.670705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d91f9-f1f4-436a-96e5-098e13a0ad3d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:02.691690 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1112/2216] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:34:02.695804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab395f-225b-4b22-9908-053ba7af7671 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:02.700389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:34:02.709293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab395f-225b-4b22-9908-053ba7af7671 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:02.715171 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdbf4d-c896-475c-9585-2520f99b2d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:02.715772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdbf4d-c896-475c-9585-2520f99b2d1e admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:02.716044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdbf4d-c896-475c-9585-2520f99b2d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:02.716310 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdbf4d-c896-475c-9585-2520f99b2d1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:02.716961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bdbf4d-c896-475c-9585-2520f99b2d1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:02.718955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bdbf4d-c896-475c-9585-2520f99b2d1e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:02.754535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1105/2217] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:34:02 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:34:02.758255 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d91f9-f1f4-436a-96e5-098e13a0ad3d admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:02.769382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d91f9-f1f4-436a-96e5-098e13a0ad3d admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:02.828079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04640250-643b-40a0-8e8f-44b346db9247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:02.828723 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04640250-643b-40a0-8e8f-44b346db9247 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:02.828997 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04640250-643b-40a0-8e8f-44b346db9247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:02.829263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04640250-643b-40a0-8e8f-44b346db9247 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:02.829908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04640250-643b-40a0-8e8f-44b346db9247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:02.831890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04640250-643b-40a0-8e8f-44b346db9247 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:02.854715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1113/2218] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:34:02 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:34:02.858885 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdbf4d-c896-475c-9585-2520f99b2d1e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:02.869966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdbf4d-c896-475c-9585-2520f99b2d1e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:02.874585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0a725-54f1-4819-b499-fda50d2d2752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:02.875187 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0a725-54f1-4819-b499-fda50d2d2752 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:02.875463 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0a725-54f1-4819-b499-fda50d2d2752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:02.875733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0a725-54f1-4819-b499-fda50d2d2752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:02.876378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0a725-54f1-4819-b499-fda50d2d2752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:02.877972 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0a725-54f1-4819-b499-fda50d2d2752 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:02.887935 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1106/2219] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:34:02.892045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04640250-643b-40a0-8e8f-44b346db9247 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:02.903206 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04640250-643b-40a0-8e8f-44b346db9247 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:02.908583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b2d69-aa0f-4b6b-acec-948e77d91182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:02.909219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b2d69-aa0f-4b6b-acec-948e77d91182 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:02.909487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b2d69-aa0f-4b6b-acec-948e77d91182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:02.909754 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b2d69-aa0f-4b6b-acec-948e77d91182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:02.910396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b2d69-aa0f-4b6b-acec-948e77d91182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:02.912114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b2d69-aa0f-4b6b-acec-948e77d91182 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:02.923754 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1114/2220] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:34:02.928202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0a725-54f1-4819-b499-fda50d2d2752 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:02.938459 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0a725-54f1-4819-b499-fda50d2d2752 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:02.943494 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c167b7-3e04-44bb-bb30-2bb385f99cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:02.944165 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c167b7-3e04-44bb-bb30-2bb385f99cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:02.944480 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c167b7-3e04-44bb-bb30-2bb385f99cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:02.944757 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c167b7-3e04-44bb-bb30-2bb385f99cfe admin admin] PATH_INFO: `/v3/projects/90900722166f434496b1dc39e096d8df/users/6841b2b7a6ca4d788835bc38c334f4c3/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:02.960007 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c167b7-3e04-44bb-bb30-2bb385f99cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=90900722166f434496b1dc39e096d8df, user_id=6841b2b7a6ca4d788835bc38c334f4c3, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:02.962730 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c167b7-3e04-44bb-bb30-2bb385f99cfe admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:02.977181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1107/2221] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:34:02 2022] PUT /identity/v3/projects/90900722166f434496b1dc39e096d8df/users/6841b2b7a6ca4d788835bc38c334f4c3/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:34:02.988459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c41b6-cacf-44fb-b3da-4c2e27b1e991 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:02.988745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c41b6-cacf-44fb-b3da-4c2e27b1e991 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:02.989020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c41b6-cacf-44fb-b3da-4c2e27b1e991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:03.022609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-870c41b6-cacf-44fb-b3da-4c2e27b1e991 None None] MFA Rules not processed for user `6841b2b7a6ca4d788835bc38c334f4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:34:03.023836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-870c41b6-cacf-44fb-b3da-4c2e27b1e991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:03.097793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1115/2222] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:34:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:34:03.110029 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c167b7-3e04-44bb-bb30-2bb385f99cfe admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:03.111909 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e5c167b7-3e04-44bb-bb30-2bb385f99cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:03.171891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c167b7-3e04-44bb-bb30-2bb385f99cfe admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:03.175883 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1660414-763d-447f-835b-50b6c2ef7676 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a2fb70c9ae4f4b81ac4de513b868cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:03.176599 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1660414-763d-447f-835b-50b6c2ef7676 service glance] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:03.176878 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1660414-763d-447f-835b-50b6c2ef7676 service glance] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:03.177147 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1660414-763d-447f-835b-50b6c2ef7676 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:03.185508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1660414-763d-447f-835b-50b6c2ef7676 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:03.186979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1660414-763d-447f-835b-50b6c2ef7676 service glance] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:03.196050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1108/2223] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:34:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:34:13.094664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:34:13.191844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=49,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:34:18.280419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a98d3-7480-40a6-b0c2-c9dca33e4473 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:18.280949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a98d3-7480-40a6-b0c2-c9dca33e4473 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:18.280949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a98d3-7480-40a6-b0c2-c9dca33e4473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:18.303160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-fc8a98d3-7480-40a6-b0c2-c9dca33e4473 None None] MFA Rules not processed for user `67d86f09c6304121be2021cb5c15e63e`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:34:18.304376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-fc8a98d3-7480-40a6-b0c2-c9dca33e4473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:18.368499 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1116/2224] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:34:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:34:28.357028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=25 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:34:31.400576 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1660414-763d-447f-835b-50b6c2ef7676 service glance] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:31.465213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1660414-763d-447f-835b-50b6c2ef7676 service glance] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:31.469992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925ccd5-f30b-460d-8cd4-7805754aa736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:31.470664 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925ccd5-f30b-460d-8cd4-7805754aa736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:31.470941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925ccd5-f30b-460d-8cd4-7805754aa736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:31.471209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925ccd5-f30b-460d-8cd4-7805754aa736 admin admin] PATH_INFO: `/v3/users/6841b2b7a6ca4d788835bc38c334f4c3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:31.472339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925ccd5-f30b-460d-8cd4-7805754aa736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6841b2b7a6ca4d788835bc38c334f4c3)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:31.474255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925ccd5-f30b-460d-8cd4-7805754aa736 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:31.525480 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9925ccd5-f30b-460d-8cd4-7805754aa736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6841b2b7a6ca4d788835bc38c334f4c3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:31.526267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9925ccd5-f30b-460d-8cd4-7805754aa736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6841b2b7a6ca4d788835bc38c334f4c3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:31.527121 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9925ccd5-f30b-460d-8cd4-7805754aa736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6841b2b7a6ca4d788835bc38c334f4c3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:31.536005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9925ccd5-f30b-460d-8cd4-7805754aa736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6841b2b7a6ca4d788835bc38c334f4c3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:31.551246 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-9925ccd5-f30b-460d-8cd4-7805754aa736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6841b2b7a6ca4d788835bc38c334f4c3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:31.567364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1109/2225] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:34:31 2022] DELETE /identity/v3/users/6841b2b7a6ca4d788835bc38c334f4c3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:34:31.572052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2b681-a212-4592-866e-e0b59f397003 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:31.572606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2b681-a212-4592-866e-e0b59f397003 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:31.572606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2b681-a212-4592-866e-e0b59f397003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:31.592113 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f1c2b681-a212-4592-866e-e0b59f397003 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:34:31.593321 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f1c2b681-a212-4592-866e-e0b59f397003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:31.653440 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1117/2226] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:34:31.665049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925ccd5-f30b-460d-8cd4-7805754aa736 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:31.667280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9925ccd5-f30b-460d-8cd4-7805754aa736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:31.727204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925ccd5-f30b-460d-8cd4-7805754aa736 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:31.731177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73745640-8754-4420-b094-230aa8064a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:31.731776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73745640-8754-4420-b094-230aa8064a43 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:31.732048 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73745640-8754-4420-b094-230aa8064a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:31.732331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73745640-8754-4420-b094-230aa8064a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:31.741491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73745640-8754-4420-b094-230aa8064a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:31.742976 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73745640-8754-4420-b094-230aa8064a43 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:31.752474 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1110/2227] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:34:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:34:32.013838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2b681-a212-4592-866e-e0b59f397003 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:32.015449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f1c2b681-a212-4592-866e-e0b59f397003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:32.028660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2b681-a212-4592-866e-e0b59f397003 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:32.034032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:32.034705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:32.035002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:32.035246 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b admin admin] PATH_INFO: `/v3/projects/90900722166f434496b1dc39e096d8df` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:32.036539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b admin admin] RBAC: Authorizing `identity:delete_project(project_id=90900722166f434496b1dc39e096d8df)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:32.038478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:32.094104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90900722166f434496b1dc39e096d8df', 'request_id': 'req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:32.100545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90900722166f434496b1dc39e096d8df', 'request_id': 'req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:32.110386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90900722166f434496b1dc39e096d8df', 'request_id': 'req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:32.120637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b admin admin] The token cache is being invalidate because project 90900722166f434496b1dc39e096d8df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:34:32.120919 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:32.121537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c1d603f3-e266-49eb-923a-d5fb3ad9f82b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:34:32.123088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1118/2228] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:34:32 2022] DELETE /identity/v3/projects/90900722166f434496b1dc39e096d8df => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:34:32.128031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f3667-0c3e-49f1-8e81-1c7f524cf8c8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:32.128309 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f3667-0c3e-49f1-8e81-1c7f524cf8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:32.128573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f3667-0c3e-49f1-8e81-1c7f524cf8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:32.147941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-852f3667-0c3e-49f1-8e81-1c7f524cf8c8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:34:32.149120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-852f3667-0c3e-49f1-8e81-1c7f524cf8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:32.166535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:34:32.210335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1111/2229] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:34:32.218359 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ee991-9a72-49e7-a9e3-84fae0879482 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:32.218805 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ee991-9a72-49e7-a9e3-84fae0879482 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:32.219088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ee991-9a72-49e7-a9e3-84fae0879482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:32.239456 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-c27ee991-9a72-49e7-a9e3-84fae0879482 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:34:32.240654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c27ee991-9a72-49e7-a9e3-84fae0879482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:32.249486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1119/2230] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:34:32.253727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f3667-0c3e-49f1-8e81-1c7f524cf8c8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:32.263656 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f3667-0c3e-49f1-8e81-1c7f524cf8c8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:32.271515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f798f6e-39c1-4fc6-a0d2-e19c7f205ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:32.272374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f798f6e-39c1-4fc6-a0d2-e19c7f205ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:32.272667 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f798f6e-39c1-4fc6-a0d2-e19c7f205ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:32.273043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f798f6e-39c1-4fc6-a0d2-e19c7f205ae3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:32.273963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f798f6e-39c1-4fc6-a0d2-e19c7f205ae3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:34:32.274346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f798f6e-39c1-4fc6-a0d2-e19c7f205ae3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:32.276769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f798f6e-39c1-4fc6-a0d2-e19c7f205ae3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:32.290207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1112/2231] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:34:32.295943 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ee991-9a72-49e7-a9e3-84fae0879482 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:32.305459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ee991-9a72-49e7-a9e3-84fae0879482 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:32.311461 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2f20a-b251-4680-ac40-a32342407c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:32.312073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2f20a-b251-4680-ac40-a32342407c8c admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:32.312350 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2f20a-b251-4680-ac40-a32342407c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:32.312617 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2f20a-b251-4680-ac40-a32342407c8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:32.313248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2f20a-b251-4680-ac40-a32342407c8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:32.315207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2f20a-b251-4680-ac40-a32342407c8c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:32.347406 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1120/2232] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:34:32 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:34:32.351634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f798f6e-39c1-4fc6-a0d2-e19c7f205ae3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:32.361803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f798f6e-39c1-4fc6-a0d2-e19c7f205ae3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:32.421970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8015841-c4ee-4acc-994e-5aa463d08a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:32.422877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8015841-c4ee-4acc-994e-5aa463d08a4d admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:32.422877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8015841-c4ee-4acc-994e-5aa463d08a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:32.423150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8015841-c4ee-4acc-994e-5aa463d08a4d admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:32.423812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8015841-c4ee-4acc-994e-5aa463d08a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:32.425760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8015841-c4ee-4acc-994e-5aa463d08a4d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:32.450207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1113/2233] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:34:32 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:34:32.451467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2f20a-b251-4680-ac40-a32342407c8c admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:32.464536 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2f20a-b251-4680-ac40-a32342407c8c admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:32.469267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9a4e1-9a08-4c62-9a06-0bbe412f1f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:32.469871 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9a4e1-9a08-4c62-9a06-0bbe412f1f83 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:32.470148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9a4e1-9a08-4c62-9a06-0bbe412f1f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:32.470421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9a4e1-9a08-4c62-9a06-0bbe412f1f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:32.471088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9a4e1-9a08-4c62-9a06-0bbe412f1f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:32.472672 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9a4e1-9a08-4c62-9a06-0bbe412f1f83 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:32.482949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1121/2234] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:34:32.487181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8015841-c4ee-4acc-994e-5aa463d08a4d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:32.497027 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8015841-c4ee-4acc-994e-5aa463d08a4d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:32.501727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b37b5-ed9d-4159-8217-d4dff7ab8e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:32.502329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b37b5-ed9d-4159-8217-d4dff7ab8e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:32.502639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b37b5-ed9d-4159-8217-d4dff7ab8e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:32.502912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b37b5-ed9d-4159-8217-d4dff7ab8e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:32.503552 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b37b5-ed9d-4159-8217-d4dff7ab8e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:32.505154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b37b5-ed9d-4159-8217-d4dff7ab8e82 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:32.515583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1114/2235] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:34:32.520164 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9a4e1-9a08-4c62-9a06-0bbe412f1f83 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:32.530465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9a4e1-9a08-4c62-9a06-0bbe412f1f83 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:32.535034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efd268-9945-4404-a791-37157fbf0fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:32.535695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efd268-9945-4404-a791-37157fbf0fa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:32.535970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efd268-9945-4404-a791-37157fbf0fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:32.536239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efd268-9945-4404-a791-37157fbf0fa6 admin admin] PATH_INFO: `/v3/projects/62ae4bf7436d4a7085d48041c81e7605/users/b951178627a54549af498f5264eec1c2/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:32.551414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1efd268-9945-4404-a791-37157fbf0fa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62ae4bf7436d4a7085d48041c81e7605, user_id=b951178627a54549af498f5264eec1c2, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:32.554059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1efd268-9945-4404-a791-37157fbf0fa6 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:32.569823 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1122/2236] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:34:32 2022] PUT /identity/v3/projects/62ae4bf7436d4a7085d48041c81e7605/users/b951178627a54549af498f5264eec1c2/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:34:32.580385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f419e-d5e9-46ad-bc6a-02c91dc3b9b8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:32.580672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f419e-d5e9-46ad-bc6a-02c91dc3b9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:32.580931 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f419e-d5e9-46ad-bc6a-02c91dc3b9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:32.610972 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4d1f419e-d5e9-46ad-bc6a-02c91dc3b9b8 None None] MFA Rules not processed for user `b951178627a54549af498f5264eec1c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:34:32.612204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4d1f419e-d5e9-46ad-bc6a-02c91dc3b9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:32.681159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1115/2237] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:34:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:34:32.692582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efd268-9945-4404-a791-37157fbf0fa6 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:32.694370 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c1efd268-9945-4404-a791-37157fbf0fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:32.718602 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:34:32.756801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efd268-9945-4404-a791-37157fbf0fa6 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:32.760878 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f4f4a-50c3-43b6-89bc-c01b3a0ac474 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:32.761453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f4f4a-50c3-43b6-89bc-c01b3a0ac474 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:32.761755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f4f4a-50c3-43b6-89bc-c01b3a0ac474 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:32.762023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f4f4a-50c3-43b6-89bc-c01b3a0ac474 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:32.770341 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f4f4a-50c3-43b6-89bc-c01b3a0ac474 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:32.771788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f4f4a-50c3-43b6-89bc-c01b3a0ac474 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:32.780795 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1123/2238] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:34:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:34:42.678150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:34:42.776755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=17 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:34:51.567841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c25ab-0603-4b43-8293-293e3c29eac2 None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:51.568341 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c25ab-0603-4b43-8293-293e3c29eac2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:51.568446 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c25ab-0603-4b43-8293-293e3c29eac2 None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:51.569783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1116/2239] 10.222.0.23 () {54 vars in 961 bytes} [Fri Sep 9 16:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:34:51.574996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cbc34-7400-4b38-aa42-5371568f97fb None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:51.575430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cbc34-7400-4b38-aa42-5371568f97fb None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:51.575583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cbc34-7400-4b38-aa42-5371568f97fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:51.596573 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f72cbc34-7400-4b38-aa42-5371568f97fb None None] MFA Rules not processed for user `784b6147de91435292e4cf3940bcad9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:34:51.597776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f72cbc34-7400-4b38-aa42-5371568f97fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:51.666065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1124/2240] 10.222.0.23 () {58 vars in 1058 bytes} [Fri Sep 9 16:34:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:34:51.674372 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c25ab-0603-4b43-8293-293e3c29eac2 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:34:51.689356 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c25ab-0603-4b43-8293-293e3c29eac2 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:34:51.694803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f3c13-ec02-4011-a513-be7dd7fcacc7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784b6147de91435292e4cf3940bcad9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:34:51.695392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f3c13-ec02-4011-a513-be7dd7fcacc7 service placement] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:34:51.695669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f3c13-ec02-4011-a513-be7dd7fcacc7 service placement] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:34:51.695939 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f3c13-ec02-4011-a513-be7dd7fcacc7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:34:51.697870 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bb5f3c13-ec02-4011-a513-be7dd7fcacc7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:34:51.709527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f3c13-ec02-4011-a513-be7dd7fcacc7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:34:51.711139 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f3c13-ec02-4011-a513-be7dd7fcacc7 service placement] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:34:51.721521 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1117/2241] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:34:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:35:01.656157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=29 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:35:01.716853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:35:11.555810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cbc34-7400-4b38-aa42-5371568f97fb None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:11.630244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cbc34-7400-4b38-aa42-5371568f97fb None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:11.635654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8b606-30af-4ab1-8d2f-d2ca3b977fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:11.636294 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8b606-30af-4ab1-8d2f-d2ca3b977fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:11.636550 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8b606-30af-4ab1-8d2f-d2ca3b977fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:11.636818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8b606-30af-4ab1-8d2f-d2ca3b977fb1 admin admin] PATH_INFO: `/v3/users/b951178627a54549af498f5264eec1c2` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:11.637980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8b606-30af-4ab1-8d2f-d2ca3b977fb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b951178627a54549af498f5264eec1c2)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:11.639943 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8b606-30af-4ab1-8d2f-d2ca3b977fb1 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:11.689764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-3fd8b606-30af-4ab1-8d2f-d2ca3b977fb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b951178627a54549af498f5264eec1c2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:11.690456 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-3fd8b606-30af-4ab1-8d2f-d2ca3b977fb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b951178627a54549af498f5264eec1c2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:11.691134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-3fd8b606-30af-4ab1-8d2f-d2ca3b977fb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b951178627a54549af498f5264eec1c2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:11.698105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-3fd8b606-30af-4ab1-8d2f-d2ca3b977fb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b951178627a54549af498f5264eec1c2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:11.709766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-3fd8b606-30af-4ab1-8d2f-d2ca3b977fb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b951178627a54549af498f5264eec1c2'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:11.724870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1125/2242] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:35:11 2022] DELETE /identity/v3/users/b951178627a54549af498f5264eec1c2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:35:11.729653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a9d43-692a-4abf-905b-b7cf0ccb9473 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:11.730195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a9d43-692a-4abf-905b-b7cf0ccb9473 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:11.730195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a9d43-692a-4abf-905b-b7cf0ccb9473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:11.749474 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ac1a9d43-692a-4abf-905b-b7cf0ccb9473 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:11.750615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ac1a9d43-692a-4abf-905b-b7cf0ccb9473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:11.810346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1118/2243] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:35:11.820321 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8b606-30af-4ab1-8d2f-d2ca3b977fb1 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:11.821716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-3fd8b606-30af-4ab1-8d2f-d2ca3b977fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:11.878424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8b606-30af-4ab1-8d2f-d2ca3b977fb1 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:11.882980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee96b1-024a-4414-8001-953a02b45d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:11.883590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee96b1-024a-4414-8001-953a02b45d24 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:11.883914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee96b1-024a-4414-8001-953a02b45d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:11.884181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee96b1-024a-4414-8001-953a02b45d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:11.893912 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee96b1-024a-4414-8001-953a02b45d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:11.895392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee96b1-024a-4414-8001-953a02b45d24 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:11.905840 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1126/2244] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:35:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:35:12.175792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a9d43-692a-4abf-905b-b7cf0ccb9473 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:12.177605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ac1a9d43-692a-4abf-905b-b7cf0ccb9473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:12.192010 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a9d43-692a-4abf-905b-b7cf0ccb9473 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:12.197006 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71afc4-084c-4b31-92a4-563f6850feb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:12.197618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71afc4-084c-4b31-92a4-563f6850feb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:12.197897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71afc4-084c-4b31-92a4-563f6850feb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:12.198170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71afc4-084c-4b31-92a4-563f6850feb5 admin admin] PATH_INFO: `/v3/projects/62ae4bf7436d4a7085d48041c81e7605` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:12.199382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71afc4-084c-4b31-92a4-563f6850feb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62ae4bf7436d4a7085d48041c81e7605)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:12.201340 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71afc4-084c-4b31-92a4-563f6850feb5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:12.253455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1c71afc4-084c-4b31-92a4-563f6850feb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62ae4bf7436d4a7085d48041c81e7605', 'request_id': 'req-1c71afc4-084c-4b31-92a4-563f6850feb5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:12.260173 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1c71afc4-084c-4b31-92a4-563f6850feb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62ae4bf7436d4a7085d48041c81e7605', 'request_id': 'req-1c71afc4-084c-4b31-92a4-563f6850feb5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:12.266487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1c71afc4-084c-4b31-92a4-563f6850feb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62ae4bf7436d4a7085d48041c81e7605', 'request_id': 'req-1c71afc4-084c-4b31-92a4-563f6850feb5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:12.280807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1c71afc4-084c-4b31-92a4-563f6850feb5 admin admin] The token cache is being invalidate because project 62ae4bf7436d4a7085d48041c81e7605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:35:12.281044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1c71afc4-084c-4b31-92a4-563f6850feb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:12.281695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1c71afc4-084c-4b31-92a4-563f6850feb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:12.283239 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1119/2245] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:35:12 2022] DELETE /identity/v3/projects/62ae4bf7436d4a7085d48041c81e7605 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:35:12.288396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b22b7-8c6c-44ec-828d-525d312b8609 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:12.288668 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b22b7-8c6c-44ec-828d-525d312b8609 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:12.288943 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b22b7-8c6c-44ec-828d-525d312b8609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:12.309855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-421b22b7-8c6c-44ec-828d-525d312b8609 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:12.311226 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-421b22b7-8c6c-44ec-828d-525d312b8609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:12.328926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:35:12.372385 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1127/2246] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:12.380813 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852488e-a0c0-4de2-9670-5768ca2cfd2a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:12.381129 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852488e-a0c0-4de2-9670-5768ca2cfd2a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:12.381408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852488e-a0c0-4de2-9670-5768ca2cfd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:12.401777 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5852488e-a0c0-4de2-9670-5768ca2cfd2a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:12.403004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5852488e-a0c0-4de2-9670-5768ca2cfd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:12.411106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1120/2247] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:12.415308 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b22b7-8c6c-44ec-828d-525d312b8609 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:12.425159 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b22b7-8c6c-44ec-828d-525d312b8609 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:12.429854 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a18c4-c800-4cb8-892a-2e0a8ab68b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:12.430455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a18c4-c800-4cb8-892a-2e0a8ab68b49 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:12.430756 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a18c4-c800-4cb8-892a-2e0a8ab68b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:12.431026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a18c4-c800-4cb8-892a-2e0a8ab68b49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:12.431705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a18c4-c800-4cb8-892a-2e0a8ab68b49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:35:12.432003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a18c4-c800-4cb8-892a-2e0a8ab68b49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:12.433601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a18c4-c800-4cb8-892a-2e0a8ab68b49 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:12.445531 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1128/2248] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:35:12.449887 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852488e-a0c0-4de2-9670-5768ca2cfd2a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:12.459423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852488e-a0c0-4de2-9670-5768ca2cfd2a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:12.465206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc32656-d2e1-493c-a5ff-1976d19ac187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:12.465801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc32656-d2e1-493c-a5ff-1976d19ac187 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:12.466080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc32656-d2e1-493c-a5ff-1976d19ac187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:12.466351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc32656-d2e1-493c-a5ff-1976d19ac187 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:12.466990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc32656-d2e1-493c-a5ff-1976d19ac187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:12.468953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc32656-d2e1-493c-a5ff-1976d19ac187 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:12.499630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1121/2249] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:35:12 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:35:12.506280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a18c4-c800-4cb8-892a-2e0a8ab68b49 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:12.521334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a18c4-c800-4cb8-892a-2e0a8ab68b49 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:12.579468 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a81a2b-f0a3-4b13-aaaa-821380b73f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:12.580108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a81a2b-f0a3-4b13-aaaa-821380b73f29 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:12.580349 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a81a2b-f0a3-4b13-aaaa-821380b73f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:12.580622 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a81a2b-f0a3-4b13-aaaa-821380b73f29 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:12.581246 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a81a2b-f0a3-4b13-aaaa-821380b73f29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:12.583310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a81a2b-f0a3-4b13-aaaa-821380b73f29 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:12.607939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1129/2250] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:35:12 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:35:12.612885 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc32656-d2e1-493c-a5ff-1976d19ac187 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:12.624328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc32656-d2e1-493c-a5ff-1976d19ac187 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:12.631061 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bee5d-e94c-48e1-b979-3a5fbf3723d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:12.631669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bee5d-e94c-48e1-b979-3a5fbf3723d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:12.631949 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bee5d-e94c-48e1-b979-3a5fbf3723d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:12.632203 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bee5d-e94c-48e1-b979-3a5fbf3723d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:12.632845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bee5d-e94c-48e1-b979-3a5fbf3723d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:12.634459 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bee5d-e94c-48e1-b979-3a5fbf3723d4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:12.645914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1122/2251] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:35:12.650326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a81a2b-f0a3-4b13-aaaa-821380b73f29 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:12.665321 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a81a2b-f0a3-4b13-aaaa-821380b73f29 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:12.673178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9725fba-7fc6-489a-a8df-57b7958a4c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:12.674025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9725fba-7fc6-489a-a8df-57b7958a4c26 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:12.674338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9725fba-7fc6-489a-a8df-57b7958a4c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:12.674630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9725fba-7fc6-489a-a8df-57b7958a4c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:12.675415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9725fba-7fc6-489a-a8df-57b7958a4c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:12.677645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9725fba-7fc6-489a-a8df-57b7958a4c26 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:12.688488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1130/2252] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:35:12.692720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bee5d-e94c-48e1-b979-3a5fbf3723d4 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:12.702691 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bee5d-e94c-48e1-b979-3a5fbf3723d4 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:12.707791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e40477-16a2-45d3-91c4-7223f421bc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:12.708504 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e40477-16a2-45d3-91c4-7223f421bc30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:12.708777 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e40477-16a2-45d3-91c4-7223f421bc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:12.709073 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e40477-16a2-45d3-91c4-7223f421bc30 admin admin] PATH_INFO: `/v3/projects/35ba7a53856149648c73617c5efdd26b/users/49351ba5b71944609e8e18539e07a112/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:12.725347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e40477-16a2-45d3-91c4-7223f421bc30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35ba7a53856149648c73617c5efdd26b, user_id=49351ba5b71944609e8e18539e07a112, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:12.728036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e40477-16a2-45d3-91c4-7223f421bc30 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:12.743667 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1123/2253] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:35:12 2022] PUT /identity/v3/projects/35ba7a53856149648c73617c5efdd26b/users/49351ba5b71944609e8e18539e07a112/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:35:12.754186 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866da5b-898a-44bf-913f-e44e0aab70e4 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:12.754465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866da5b-898a-44bf-913f-e44e0aab70e4 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:12.754772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866da5b-898a-44bf-913f-e44e0aab70e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:12.785672 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-3866da5b-898a-44bf-913f-e44e0aab70e4 None None] MFA Rules not processed for user `49351ba5b71944609e8e18539e07a112`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:12.787010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-3866da5b-898a-44bf-913f-e44e0aab70e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:12.861097 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1131/2254] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:35:12 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:12.872781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e40477-16a2-45d3-91c4-7223f421bc30 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:12.874670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-33e40477-16a2-45d3-91c4-7223f421bc30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:12.897713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:35:12.934806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e40477-16a2-45d3-91c4-7223f421bc30 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:12.938844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cb3c3-b892-4a2a-855d-36353f796f59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:12.939454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cb3c3-b892-4a2a-855d-36353f796f59 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:12.939725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cb3c3-b892-4a2a-855d-36353f796f59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:12.939991 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cb3c3-b892-4a2a-855d-36353f796f59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:12.948695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406cb3c3-b892-4a2a-855d-36353f796f59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:12.950174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406cb3c3-b892-4a2a-855d-36353f796f59 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:12.959745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1124/2255] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:35:12 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:35:13.607333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6ae34-5152-422e-80ca-066624ae424d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:13.607869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6ae34-5152-422e-80ca-066624ae424d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:13.608141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6ae34-5152-422e-80ca-066624ae424d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:13.627673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-5aa6ae34-5152-422e-80ca-066624ae424d None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:13.628852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5aa6ae34-5152-422e-80ca-066624ae424d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:13.641333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:35:13.687767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1132/2256] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:35:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:13.743089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cb3c3-b892-4a2a-855d-36353f796f59 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:13.744585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-406cb3c3-b892-4a2a-855d-36353f796f59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:13.824841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cb3c3-b892-4a2a-855d-36353f796f59 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:13.830103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2a501-c474-4f0a-9cd8-93b4388c45ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:13.830743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a501-c474-4f0a-9cd8-93b4388c45ad service swift] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:13.831017 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a501-c474-4f0a-9cd8-93b4388c45ad service swift] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:13.831277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a501-c474-4f0a-9cd8-93b4388c45ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:13.839593 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2a501-c474-4f0a-9cd8-93b4388c45ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:13.841202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2a501-c474-4f0a-9cd8-93b4388c45ad service swift] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:13.849260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1125/2257] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:35:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:35:20.351168 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6ae34-5152-422e-80ca-066624ae424d None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:20.459910 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6ae34-5152-422e-80ca-066624ae424d None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:20.464894 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272d47b-c253-43b4-bf63-e9eaf73e3264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:20.465794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272d47b-c253-43b4-bf63-e9eaf73e3264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:20.466144 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272d47b-c253-43b4-bf63-e9eaf73e3264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:20.466475 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272d47b-c253-43b4-bf63-e9eaf73e3264 admin admin] PATH_INFO: `/v3/users/49351ba5b71944609e8e18539e07a112` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:20.467837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272d47b-c253-43b4-bf63-e9eaf73e3264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49351ba5b71944609e8e18539e07a112)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:20.472781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272d47b-c253-43b4-bf63-e9eaf73e3264 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:20.554699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4272d47b-c253-43b4-bf63-e9eaf73e3264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49351ba5b71944609e8e18539e07a112'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:20.555406 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4272d47b-c253-43b4-bf63-e9eaf73e3264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49351ba5b71944609e8e18539e07a112'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:20.556088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4272d47b-c253-43b4-bf63-e9eaf73e3264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49351ba5b71944609e8e18539e07a112'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:20.564299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4272d47b-c253-43b4-bf63-e9eaf73e3264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49351ba5b71944609e8e18539e07a112'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:20.594672 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4272d47b-c253-43b4-bf63-e9eaf73e3264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49351ba5b71944609e8e18539e07a112'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:20.615838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1133/2258] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:35:20 2022] DELETE /identity/v3/users/49351ba5b71944609e8e18539e07a112 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:35:20.620487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425d02a-a372-4e05-ac41-54bed11af76e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:20.621053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425d02a-a372-4e05-ac41-54bed11af76e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:20.621053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425d02a-a372-4e05-ac41-54bed11af76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:20.654303 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1425d02a-a372-4e05-ac41-54bed11af76e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:20.656870 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1425d02a-a372-4e05-ac41-54bed11af76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:20.802439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1126/2259] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:20.813513 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272d47b-c253-43b4-bf63-e9eaf73e3264 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:20.815218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-4272d47b-c253-43b4-bf63-e9eaf73e3264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:20.947260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:35:20.964161 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272d47b-c253-43b4-bf63-e9eaf73e3264 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:20.968239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36555e-7b14-4135-9c01-dbfc8bbc8364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:20.969375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36555e-7b14-4135-9c01-dbfc8bbc8364 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:20.969375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36555e-7b14-4135-9c01-dbfc8bbc8364 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:20.969586 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36555e-7b14-4135-9c01-dbfc8bbc8364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:20.989262 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a36555e-7b14-4135-9c01-dbfc8bbc8364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:20.991265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a36555e-7b14-4135-9c01-dbfc8bbc8364 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:21.026089 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1134/2260] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:35:21.573618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425d02a-a372-4e05-ac41-54bed11af76e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:21.579687 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1425d02a-a372-4e05-ac41-54bed11af76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:21.600347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425d02a-a372-4e05-ac41-54bed11af76e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:21.608920 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59ae78-1785-4044-b8c2-3439e56d1a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:21.610732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59ae78-1785-4044-b8c2-3439e56d1a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:21.611228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59ae78-1785-4044-b8c2-3439e56d1a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:21.611429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59ae78-1785-4044-b8c2-3439e56d1a15 admin admin] PATH_INFO: `/v3/projects/35ba7a53856149648c73617c5efdd26b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:21.614309 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59ae78-1785-4044-b8c2-3439e56d1a15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35ba7a53856149648c73617c5efdd26b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:21.617260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59ae78-1785-4044-b8c2-3439e56d1a15 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:21.680208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:35:21.681830 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0c59ae78-1785-4044-b8c2-3439e56d1a15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35ba7a53856149648c73617c5efdd26b', 'request_id': 'req-0c59ae78-1785-4044-b8c2-3439e56d1a15'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:21.689989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0c59ae78-1785-4044-b8c2-3439e56d1a15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35ba7a53856149648c73617c5efdd26b', 'request_id': 'req-0c59ae78-1785-4044-b8c2-3439e56d1a15'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:21.702058 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0c59ae78-1785-4044-b8c2-3439e56d1a15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35ba7a53856149648c73617c5efdd26b', 'request_id': 'req-0c59ae78-1785-4044-b8c2-3439e56d1a15'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:21.727713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0c59ae78-1785-4044-b8c2-3439e56d1a15 admin admin] The token cache is being invalidate because project 35ba7a53856149648c73617c5efdd26b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:35:21.727998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0c59ae78-1785-4044-b8c2-3439e56d1a15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:21.728890 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-0c59ae78-1785-4044-b8c2-3439e56d1a15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:35:21.731205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1127/2261] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:35:21 2022] DELETE /identity/v3/projects/35ba7a53856149648c73617c5efdd26b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:35:21.738560 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb2c68-00c6-450d-8dab-8a82f06e9bbc None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:21.739683 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb2c68-00c6-450d-8dab-8a82f06e9bbc None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:21.740041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb2c68-00c6-450d-8dab-8a82f06e9bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:21.767257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-b0cb2c68-00c6-450d-8dab-8a82f06e9bbc None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:21.769147 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b0cb2c68-00c6-450d-8dab-8a82f06e9bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:21.884261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1135/2262] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:21.886331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b3c5f-c2a0-4f95-b102-043f81732a95 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:21.886648 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b3c5f-c2a0-4f95-b102-043f81732a95 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:21.886925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b3c5f-c2a0-4f95-b102-043f81732a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:21.911536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-045b3c5f-c2a0-4f95-b102-043f81732a95 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:21.913054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-045b3c5f-c2a0-4f95-b102-043f81732a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:21.926530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1128/2263] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:21.930997 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb2c68-00c6-450d-8dab-8a82f06e9bbc None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:21.943472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb2c68-00c6-450d-8dab-8a82f06e9bbc None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:21.949036 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912fddbb-6a86-4f3a-82de-9973f1257d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:21.949662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fddbb-6a86-4f3a-82de-9973f1257d72 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:21.949941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fddbb-6a86-4f3a-82de-9973f1257d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:21.950210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fddbb-6a86-4f3a-82de-9973f1257d72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:21.950956 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912fddbb-6a86-4f3a-82de-9973f1257d72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:35:21.951260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912fddbb-6a86-4f3a-82de-9973f1257d72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:21.952870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912fddbb-6a86-4f3a-82de-9973f1257d72 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:21.969421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1136/2264] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:35:21.971717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b3c5f-c2a0-4f95-b102-043f81732a95 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:21.984483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b3c5f-c2a0-4f95-b102-043f81732a95 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:21.997027 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebc6b2-f6ab-4bf4-8ff3-bbfd5eb491ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:21.997903 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebc6b2-f6ab-4bf4-8ff3-bbfd5eb491ed admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:21.998326 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebc6b2-f6ab-4bf4-8ff3-bbfd5eb491ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:21.998793 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebc6b2-f6ab-4bf4-8ff3-bbfd5eb491ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:21.999696 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebc6b2-f6ab-4bf4-8ff3-bbfd5eb491ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:22.001765 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebc6b2-f6ab-4bf4-8ff3-bbfd5eb491ed admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:22.056964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1129/2265] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:35:21 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:35:22.061149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912fddbb-6a86-4f3a-82de-9973f1257d72 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:22.071738 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912fddbb-6a86-4f3a-82de-9973f1257d72 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:22.202733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90982bc7-a979-434d-ac95-4fb25b4bda85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:22.203877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90982bc7-a979-434d-ac95-4fb25b4bda85 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:22.204624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90982bc7-a979-434d-ac95-4fb25b4bda85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:22.204979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90982bc7-a979-434d-ac95-4fb25b4bda85 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:22.205744 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90982bc7-a979-434d-ac95-4fb25b4bda85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:22.207911 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90982bc7-a979-434d-ac95-4fb25b4bda85 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:22.242177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1137/2266] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:35:22 2022] POST /identity/v3/users => generated 287 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:35:22.243510 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebc6b2-f6ab-4bf4-8ff3-bbfd5eb491ed admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:22.257560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebc6b2-f6ab-4bf4-8ff3-bbfd5eb491ed admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:22.271654 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a40319-d2ec-4ed4-ac92-fbc8ae02b8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:22.272611 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a40319-d2ec-4ed4-ac92-fbc8ae02b8cd admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:22.273000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a40319-d2ec-4ed4-ac92-fbc8ae02b8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:22.273420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a40319-d2ec-4ed4-ac92-fbc8ae02b8cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:22.274423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a40319-d2ec-4ed4-ac92-fbc8ae02b8cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:22.277115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a40319-d2ec-4ed4-ac92-fbc8ae02b8cd admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:22.292230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1130/2267] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:35:22.296306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90982bc7-a979-434d-ac95-4fb25b4bda85 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:22.321408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90982bc7-a979-434d-ac95-4fb25b4bda85 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:22.361752 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae1e59-3dd6-463f-ac46-5cf702c09c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:22.362823 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae1e59-3dd6-463f-ac46-5cf702c09c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:22.363308 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae1e59-3dd6-463f-ac46-5cf702c09c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:22.363876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae1e59-3dd6-463f-ac46-5cf702c09c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:22.364748 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae1e59-3dd6-463f-ac46-5cf702c09c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:22.367334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae1e59-3dd6-463f-ac46-5cf702c09c94 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:22.385061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1138/2268] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:35:22.388994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a40319-d2ec-4ed4-ac92-fbc8ae02b8cd admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:22.411060 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a40319-d2ec-4ed4-ac92-fbc8ae02b8cd admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:22.419470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35f6d9-9f09-4061-82ee-d8ff24f6e00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:22.420431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35f6d9-9f09-4061-82ee-d8ff24f6e00b admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:22.420913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35f6d9-9f09-4061-82ee-d8ff24f6e00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:22.421396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35f6d9-9f09-4061-82ee-d8ff24f6e00b admin admin] PATH_INFO: `/v3/projects/1ba803ac5e71467f853215e19a7f8867/users/208daa6413d34e4496e278fb3fc251a5/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:22.446212 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35f6d9-9f09-4061-82ee-d8ff24f6e00b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba803ac5e71467f853215e19a7f8867, user_id=208daa6413d34e4496e278fb3fc251a5, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:22.451734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35f6d9-9f09-4061-82ee-d8ff24f6e00b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:22.470548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1131/2269] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:35:22 2022] PUT /identity/v3/projects/1ba803ac5e71467f853215e19a7f8867/users/208daa6413d34e4496e278fb3fc251a5/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:35:22.475559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0b970-24a5-4e50-9e0e-549de6f489fb None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:22.475870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0b970-24a5-4e50-9e0e-549de6f489fb None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:22.476151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0b970-24a5-4e50-9e0e-549de6f489fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:22.508344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-dcb0b970-24a5-4e50-9e0e-549de6f489fb None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:22.510715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-dcb0b970-24a5-4e50-9e0e-549de6f489fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:22.597024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:35:22.625815 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1139/2270] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:22.636484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35f6d9-9f09-4061-82ee-d8ff24f6e00b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:22.638547 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dc35f6d9-9f09-4061-82ee-d8ff24f6e00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:22.709745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35f6d9-9f09-4061-82ee-d8ff24f6e00b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:22.713996 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8aa80-15ee-4d62-85d3-d5e652141e60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:22.714925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8aa80-15ee-4d62-85d3-d5e652141e60 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:22.715298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8aa80-15ee-4d62-85d3-d5e652141e60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:22.715631 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8aa80-15ee-4d62-85d3-d5e652141e60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:22.725582 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8aa80-15ee-4d62-85d3-d5e652141e60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:22.727528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8aa80-15ee-4d62-85d3-d5e652141e60 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:22.742403 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1132/2271] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:35:25.005152 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a77a6-7631-4c2a-a6df-189f9101ab8a None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:25.005567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a77a6-7631-4c2a-a6df-189f9101ab8a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:25.005698 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a77a6-7631-4c2a-a6df-189f9101ab8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:25.031962 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-8a2a77a6-7631-4c2a-a6df-189f9101ab8a None None] MFA Rules not processed for user `53c0721dc18343b18993ff5af098841d`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:25.033213 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8a2a77a6-7631-4c2a-a6df-189f9101ab8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:25.117033 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1140/2272] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:35:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:28.054770 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a45f4-dbf1-43e8-bd8f-14c897a88521 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:28.055261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a45f4-dbf1-43e8-bd8f-14c897a88521 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:28.055541 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a45f4-dbf1-43e8-bd8f-14c897a88521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:28.109490 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-906a45f4-dbf1-43e8-bd8f-14c897a88521 None None] MFA Rules not processed for user `208daa6413d34e4496e278fb3fc251a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:28.110919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-906a45f4-dbf1-43e8-bd8f-14c897a88521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:28.259339 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a77a6-7631-4c2a-a6df-189f9101ab8a None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:28.270932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1133/2273] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:35:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:28.275651 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a77a6-7631-4c2a-a6df-189f9101ab8a None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:28.291132 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17769bf7-557e-455c-9839-34020ce8249d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:28.291791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17769bf7-557e-455c-9839-34020ce8249d admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:28.292069 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17769bf7-557e-455c-9839-34020ce8249d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:28.292334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17769bf7-557e-455c-9839-34020ce8249d admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:28.292977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17769bf7-557e-455c-9839-34020ce8249d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:28.295532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17769bf7-557e-455c-9839-34020ce8249d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:28.352875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1141/2274] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:35:28 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:35:28.353959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a45f4-dbf1-43e8-bd8f-14c897a88521 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:28.372588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:35:28.377634 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a45f4-dbf1-43e8-bd8f-14c897a88521 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:28.438850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62f315-3b06-4a7e-bdfa-2f369de2603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:28.439474 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62f315-3b06-4a7e-bdfa-2f369de2603c admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:28.439773 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62f315-3b06-4a7e-bdfa-2f369de2603c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:28.440037 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62f315-3b06-4a7e-bdfa-2f369de2603c admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:28.440696 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62f315-3b06-4a7e-bdfa-2f369de2603c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:28.442764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62f315-3b06-4a7e-bdfa-2f369de2603c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:28.465176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1134/2275] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:35:28 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:35:28.469294 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17769bf7-557e-455c-9839-34020ce8249d admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:28.480675 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17769bf7-557e-455c-9839-34020ce8249d admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:28.485512 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b646a31-2c50-42ea-b273-e3a566cba3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:28.486103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b646a31-2c50-42ea-b273-e3a566cba3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:28.486344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b646a31-2c50-42ea-b273-e3a566cba3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:28.486807 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b646a31-2c50-42ea-b273-e3a566cba3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:28.487937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b646a31-2c50-42ea-b273-e3a566cba3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:28.490137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b646a31-2c50-42ea-b273-e3a566cba3f6 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:28.502681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1142/2276] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:35:28.507603 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62f315-3b06-4a7e-bdfa-2f369de2603c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:28.519117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62f315-3b06-4a7e-bdfa-2f369de2603c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:28.524536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be4fd6-daad-4d35-af7c-bd2c83f9a4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:28.525291 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be4fd6-daad-4d35-af7c-bd2c83f9a4c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:28.525654 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be4fd6-daad-4d35-af7c-bd2c83f9a4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:28.526056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be4fd6-daad-4d35-af7c-bd2c83f9a4c3 admin admin] PATH_INFO: `/v3/projects/e19301f6dc8c4ed1815a8ab00125c8e5/users/b1361af4674349998ad90ce0e79391c5/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:28.547001 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be4fd6-daad-4d35-af7c-bd2c83f9a4c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19301f6dc8c4ed1815a8ab00125c8e5, user_id=b1361af4674349998ad90ce0e79391c5, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:28.549886 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be4fd6-daad-4d35-af7c-bd2c83f9a4c3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:28.568385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1135/2277] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:35:28 2022] PUT /identity/v3/projects/e19301f6dc8c4ed1815a8ab00125c8e5/users/b1361af4674349998ad90ce0e79391c5/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:35:28.572492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b646a31-2c50-42ea-b273-e3a566cba3f6 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:28.582798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b646a31-2c50-42ea-b273-e3a566cba3f6 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:28.657178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba4258-7aea-4f69-ac10-f1e6603ec0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:28.657859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba4258-7aea-4f69-ac10-f1e6603ec0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:28.658230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba4258-7aea-4f69-ac10-f1e6603ec0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:28.658443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba4258-7aea-4f69-ac10-f1e6603ec0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:28.659125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba4258-7aea-4f69-ac10-f1e6603ec0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:28.660837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba4258-7aea-4f69-ac10-f1e6603ec0d7 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:28.674896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1143/2278] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:35:28.678941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be4fd6-daad-4d35-af7c-bd2c83f9a4c3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:28.689622 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be4fd6-daad-4d35-af7c-bd2c83f9a4c3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:28.695192 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92338e4d-14b7-41df-88ed-5ffd0e02a7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:28.696092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92338e4d-14b7-41df-88ed-5ffd0e02a7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:28.696567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92338e4d-14b7-41df-88ed-5ffd0e02a7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:28.697035 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92338e4d-14b7-41df-88ed-5ffd0e02a7bc admin admin] PATH_INFO: `/v3/projects/e19301f6dc8c4ed1815a8ab00125c8e5/users/b1361af4674349998ad90ce0e79391c5/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:28.701203 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92338e4d-14b7-41df-88ed-5ffd0e02a7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19301f6dc8c4ed1815a8ab00125c8e5, user_id=b1361af4674349998ad90ce0e79391c5, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:28.712532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92338e4d-14b7-41df-88ed-5ffd0e02a7bc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:28.739749 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1136/2279] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:35:28 2022] PUT /identity/v3/projects/e19301f6dc8c4ed1815a8ab00125c8e5/users/b1361af4674349998ad90ce0e79391c5/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:35:29.767281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85802025-62dc-408b-a59d-82ffca9f7a20 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:29.767742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85802025-62dc-408b-a59d-82ffca9f7a20 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:29.768207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85802025-62dc-408b-a59d-82ffca9f7a20 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:29.770200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1144/2280] 10.222.0.23 () {54 vars in 921 bytes} [Fri Sep 9 16:35:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:35:29.777395 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfebbc-9e4f-45da-9915-90a31980daea None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:29.777916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfebbc-9e4f-45da-9915-90a31980daea None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:29.777916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfebbc-9e4f-45da-9915-90a31980daea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:29.818675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c1cfebbc-9e4f-45da-9915-90a31980daea None None] MFA Rules not processed for user `152a41ad598b4b01b1881d928c53c848`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:29.819939 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c1cfebbc-9e4f-45da-9915-90a31980daea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:29.960107 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1137/2281] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Sep 9 16:35:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:29.972127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85802025-62dc-408b-a59d-82ffca9f7a20 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:29.974947 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-85802025-62dc-408b-a59d-82ffca9f7a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:30.100421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85802025-62dc-408b-a59d-82ffca9f7a20 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:30.108095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e6b88-48ab-4898-98c3-90ec8e6f16c1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a2fb70c9ae4f4b81ac4de513b868cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:30.109388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e6b88-48ab-4898-98c3-90ec8e6f16c1 service glance] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:30.109674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e6b88-48ab-4898-98c3-90ec8e6f16c1 service glance] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:30.109949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e6b88-48ab-4898-98c3-90ec8e6f16c1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:30.143133 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e6b88-48ab-4898-98c3-90ec8e6f16c1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:30.148243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e6b88-48ab-4898-98c3-90ec8e6f16c1 service glance] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:30.181846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1145/2282] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:35:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:35:30.333819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d558621-2359-4569-8de0-6e457ace7a7e None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:30.334656 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d558621-2359-4569-8de0-6e457ace7a7e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:30.335081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d558621-2359-4569-8de0-6e457ace7a7e None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:30.337015 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1138/2283] 10.222.0.23 () {54 vars in 921 bytes} [Fri Sep 9 16:35:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:35:30.345217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdde671-fe58-4dbb-a936-27f6c977fb26 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:30.345505 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdde671-fe58-4dbb-a936-27f6c977fb26 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:30.345777 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdde671-fe58-4dbb-a936-27f6c977fb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:30.354179 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:35:30.383708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-cfdde671-fe58-4dbb-a936-27f6c977fb26 None None] MFA Rules not processed for user `aa65b6854dc54e289cc3e8e0b3746dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:30.386784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cfdde671-fe58-4dbb-a936-27f6c977fb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:30.469540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1146/2284] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Sep 9 16:35:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:30.510925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d558621-2359-4569-8de0-6e457ace7a7e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:30.514778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2d558621-2359-4569-8de0-6e457ace7a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:30.544525 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d558621-2359-4569-8de0-6e457ace7a7e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:30.549795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada0a99-050b-48e9-86d7-afed9d8c2b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:30.550695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada0a99-050b-48e9-86d7-afed9d8c2b1b service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:30.551286 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada0a99-050b-48e9-86d7-afed9d8c2b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:30.551770 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada0a99-050b-48e9-86d7-afed9d8c2b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:30.566636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada0a99-050b-48e9-86d7-afed9d8c2b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:30.570041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada0a99-050b-48e9-86d7-afed9d8c2b1b service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:30.582025 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1139/2285] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:35:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:35:30.907045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df4e02-aa8f-4a3e-9945-9ce2e4684664 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:30.908209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df4e02-aa8f-4a3e-9945-9ce2e4684664 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:30.908660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df4e02-aa8f-4a3e-9945-9ce2e4684664 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:30.910206 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1147/2286] 10.222.0.23 () {54 vars in 921 bytes} [Fri Sep 9 16:35:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:35:30.928775 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555af9-27e6-478c-8ad9-bdbc25e9f8f2 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:30.929888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555af9-27e6-478c-8ad9-bdbc25e9f8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:30.930329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555af9-27e6-478c-8ad9-bdbc25e9f8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:30.961674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c7555af9-27e6-478c-8ad9-bdbc25e9f8f2 None None] MFA Rules not processed for user `152a41ad598b4b01b1881d928c53c848`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:30.964049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c7555af9-27e6-478c-8ad9-bdbc25e9f8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:30.992397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1140/2287] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Sep 9 16:35:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:31.012094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df4e02-aa8f-4a3e-9945-9ce2e4684664 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:31.028834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df4e02-aa8f-4a3e-9945-9ce2e4684664 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:31.039382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b00903-b4a5-47e9-9f45-25b09c3a860c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:31.040021 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b00903-b4a5-47e9-9f45-25b09c3a860c service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:31.040298 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b00903-b4a5-47e9-9f45-25b09c3a860c service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:31.040569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b00903-b4a5-47e9-9f45-25b09c3a860c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:31.063134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b00903-b4a5-47e9-9f45-25b09c3a860c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:31.064528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b00903-b4a5-47e9-9f45-25b09c3a860c service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:31.101012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1148/2288] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:35:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:35:37.088408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128e6cb-5c10-4ee3-b495-4749c3ec0276 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:37.088865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128e6cb-5c10-4ee3-b495-4749c3ec0276 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:37.088982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128e6cb-5c10-4ee3-b495-4749c3ec0276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:37.119539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-9128e6cb-5c10-4ee3-b495-4749c3ec0276 None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:37.120766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9128e6cb-5c10-4ee3-b495-4749c3ec0276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:37.150131 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:35:37.192036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1141/2289] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:35:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:37.211159 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b00903-b4a5-47e9-9f45-25b09c3a860c service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:37.219158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f7b00903-b4a5-47e9-9f45-25b09c3a860c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:37.343204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c6e81-ac99-45df-b0fd-6157645104e8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:37.344332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c6e81-ac99-45df-b0fd-6157645104e8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:37.344781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c6e81-ac99-45df-b0fd-6157645104e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:37.357439 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b00903-b4a5-47e9-9f45-25b09c3a860c service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:37.362980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757257c-15c1-40d1-91fa-8adc1801d40c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:37.363907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757257c-15c1-40d1-91fa-8adc1801d40c service swift] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:37.364398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757257c-15c1-40d1-91fa-8adc1801d40c service swift] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:37.364866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757257c-15c1-40d1-91fa-8adc1801d40c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:37.373776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757257c-15c1-40d1-91fa-8adc1801d40c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:37.375878 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757257c-15c1-40d1-91fa-8adc1801d40c service swift] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:37.386049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1149/2290] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:35:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:35:37.397053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-948c6e81-ac99-45df-b0fd-6157645104e8 None None] MFA Rules not processed for user `b1361af4674349998ad90ce0e79391c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:37.408178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-948c6e81-ac99-45df-b0fd-6157645104e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:37.648394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1142/2291] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:35:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:37.662959 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757257c-15c1-40d1-91fa-8adc1801d40c service swift] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:37.664403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b757257c-15c1-40d1-91fa-8adc1801d40c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:37.691229 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757257c-15c1-40d1-91fa-8adc1801d40c service swift] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:37.700672 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38607b88-f79d-48f8-ad84-bed23e70e178 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:37.701552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38607b88-f79d-48f8-ad84-bed23e70e178 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:37.702134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38607b88-f79d-48f8-ad84-bed23e70e178 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:37.702703 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38607b88-f79d-48f8-ad84-bed23e70e178 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:37.831596 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38607b88-f79d-48f8-ad84-bed23e70e178 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:37.833282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38607b88-f79d-48f8-ad84-bed23e70e178 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:37.859252 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1150/2292] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:35:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:35:39.884907 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f92da1-207e-4e1a-9206-4f882e8ca7c4 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:39.885386 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f92da1-207e-4e1a-9206-4f882e8ca7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:39.885494 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f92da1-207e-4e1a-9206-4f882e8ca7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:39.920205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-14f92da1-207e-4e1a-9206-4f882e8ca7c4 None None] MFA Rules not processed for user `208daa6413d34e4496e278fb3fc251a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:39.923763 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-14f92da1-207e-4e1a-9206-4f882e8ca7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:39.946972 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1143/2293] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:35:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:47.846798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=88 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:35:49.908618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=49 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:35:51.671089 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7c5b-815b-4c8b-909c-322cf5e9a2a4 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:51.671618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7c5b-815b-4c8b-909c-322cf5e9a2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:51.671618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7c5b-815b-4c8b-909c-322cf5e9a2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:51.695976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-b18f7c5b-815b-4c8b-909c-322cf5e9a2a4 None None] MFA Rules not processed for user `67d86f09c6304121be2021cb5c15e63e`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:51.697116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b18f7c5b-815b-4c8b-909c-322cf5e9a2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:51.759941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1151/2294] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:35:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:59.152373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bb300-cc3c-4776-a6f4-ded7c879ec31 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:59.153199 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bb300-cc3c-4776-a6f4-ded7c879ec31 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:59.153507 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bb300-cc3c-4776-a6f4-ded7c879ec31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:59.179466 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7c6bb300-cc3c-4776-a6f4-ded7c879ec31 None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:35:59.180685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7c6bb300-cc3c-4776-a6f4-ded7c879ec31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:35:59.189311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1144/2295] 10.222.0.23 () {58 vars in 967 bytes} [Fri Sep 9 16:35:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:35:59.202012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f7c5b-815b-4c8b-909c-322cf5e9a2a4 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:35:59.218733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f7c5b-815b-4c8b-909c-322cf5e9a2a4 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:35:59.224217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdbcdf-2a13-489c-b406-de571987c569 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:35:59.224846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdbcdf-2a13-489c-b406-de571987c569 service swift] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:35:59.225118 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdbcdf-2a13-489c-b406-de571987c569 service swift] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:35:59.225384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdbcdf-2a13-489c-b406-de571987c569 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:35:59.239405 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdbcdf-2a13-489c-b406-de571987c569 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:35:59.241124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdbcdf-2a13-489c-b406-de571987c569 service swift] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:35:59.253567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1152/2296] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:35:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:36:09.172711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:36:09.250291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=31 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:36:15.761233 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bb300-cc3c-4776-a6f4-ded7c879ec31 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:36:15.771517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bb300-cc3c-4776-a6f4-ded7c879ec31 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:36:15.775577 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42eb02-79c7-4a6a-a75f-cac198662afb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:36:15.776177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42eb02-79c7-4a6a-a75f-cac198662afb service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:36:15.776454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42eb02-79c7-4a6a-a75f-cac198662afb service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:36:15.776720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42eb02-79c7-4a6a-a75f-cac198662afb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:36:15.786266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42eb02-79c7-4a6a-a75f-cac198662afb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:36:15.787733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42eb02-79c7-4a6a-a75f-cac198662afb service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:36:15.797934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1145/2297] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:36:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:36:25.793469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:36:28.073964 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6454c-9af2-4476-9d3e-754164664d03 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:36:28.074923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6454c-9af2-4476-9d3e-754164664d03 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:36:28.075314 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6454c-9af2-4476-9d3e-754164664d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:36:28.096505 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-13e6454c-9af2-4476-9d3e-754164664d03 None None] MFA Rules not processed for user `d095ed814b9e47e08d6b5a4fd2901c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:36:28.097717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-13e6454c-9af2-4476-9d3e-754164664d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:36:28.154292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1153/2298] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:36:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:36:38.147601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=25 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:36:41.208511 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42eb02-79c7-4a6a-a75f-cac198662afb service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:36:41.217614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42eb02-79c7-4a6a-a75f-cac198662afb service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:36:41.221284 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8963bb-e7ed-43ea-bd31-a89b3dfc0b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:36:41.221887 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8963bb-e7ed-43ea-bd31-a89b3dfc0b04 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:36:41.222161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8963bb-e7ed-43ea-bd31-a89b3dfc0b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:36:41.222481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8963bb-e7ed-43ea-bd31-a89b3dfc0b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:36:41.224274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ed8963bb-e7ed-43ea-bd31-a89b3dfc0b04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:36:41.239554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8963bb-e7ed-43ea-bd31-a89b3dfc0b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:36:41.241308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8963bb-e7ed-43ea-bd31-a89b3dfc0b04 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:36:41.254284 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1146/2299] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:36:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:36:46.094214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6454c-9af2-4476-9d3e-754164664d03 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:36:46.104021 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6454c-9af2-4476-9d3e-754164664d03 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:36:46.107608 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc8aaa-b286-4b07-86e1-80e0bfaa1f16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:36:46.108217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc8aaa-b286-4b07-86e1-80e0bfaa1f16 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:36:46.108522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc8aaa-b286-4b07-86e1-80e0bfaa1f16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:36:46.108811 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc8aaa-b286-4b07-86e1-80e0bfaa1f16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:36:46.110614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cfbc8aaa-b286-4b07-86e1-80e0bfaa1f16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:36:46.123983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc8aaa-b286-4b07-86e1-80e0bfaa1f16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:36:46.125390 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc8aaa-b286-4b07-86e1-80e0bfaa1f16 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:36:46.136954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1154/2300] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:36:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:36:51.248262 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:36:56.131545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=6 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:36:57.138652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8963bb-e7ed-43ea-bd31-a89b3dfc0b04 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:36:57.141713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ed8963bb-e7ed-43ea-bd31-a89b3dfc0b04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:36:57.163843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8963bb-e7ed-43ea-bd31-a89b3dfc0b04 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:36:57.171246 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2a622-71dc-458e-819e-e853167d5725 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:36:57.172383 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2a622-71dc-458e-819e-e853167d5725 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:36:57.173538 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2a622-71dc-458e-819e-e853167d5725 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:36:57.174750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2a622-71dc-458e-819e-e853167d5725 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:36:57.176889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-80d2a622-71dc-458e-819e-e853167d5725 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:36:57.321830 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2a622-71dc-458e-819e-e853167d5725 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:36:57.325013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2a622-71dc-458e-819e-e853167d5725 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:36:57.340274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1147/2301] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:36:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:36:59.967497 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc8aaa-b286-4b07-86e1-80e0bfaa1f16 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:36:59.981831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc8aaa-b286-4b07-86e1-80e0bfaa1f16 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:36:59.988938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9286b-7578-4cd8-bf0c-a49253608d49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:36:59.989530 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9286b-7578-4cd8-bf0c-a49253608d49 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:36:59.989802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9286b-7578-4cd8-bf0c-a49253608d49 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:36:59.990062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9286b-7578-4cd8-bf0c-a49253608d49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:36:59.992094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-61f9286b-7578-4cd8-bf0c-a49253608d49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:00.002533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9286b-7578-4cd8-bf0c-a49253608d49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:00.004131 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9286b-7578-4cd8-bf0c-a49253608d49 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:00.013200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1155/2302] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:36:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:04.835840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2a622-71dc-458e-819e-e853167d5725 service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:04.854062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2a622-71dc-458e-819e-e853167d5725 service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:04.863889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b958d6d-c9d5-4d15-a391-2438b43ba117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:04.865312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b958d6d-c9d5-4d15-a391-2438b43ba117 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:04.866248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b958d6d-c9d5-4d15-a391-2438b43ba117 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:04.866797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b958d6d-c9d5-4d15-a391-2438b43ba117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:04.869828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5b958d6d-c9d5-4d15-a391-2438b43ba117 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:04.986758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b958d6d-c9d5-4d15-a391-2438b43ba117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:04.988255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b958d6d-c9d5-4d15-a391-2438b43ba117 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:05.002423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1148/2303] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:06.451710 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02407d-7395-4783-8b3f-bcfc2c29b8f5 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:06.452243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02407d-7395-4783-8b3f-bcfc2c29b8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:06.452243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02407d-7395-4783-8b3f-bcfc2c29b8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:06.473579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-2a02407d-7395-4783-8b3f-bcfc2c29b8f5 None None] MFA Rules not processed for user `f8bd900e231242c3ae67432d26d089da`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:06.475030 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2a02407d-7395-4783-8b3f-bcfc2c29b8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:06.537083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1156/2304] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:06.874247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b958d6d-c9d5-4d15-a391-2438b43ba117 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:06.883414 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b958d6d-c9d5-4d15-a391-2438b43ba117 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:06.888364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91610a-e743-4987-9a97-2c7b4ceb660f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:06.889001 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91610a-e743-4987-9a97-2c7b4ceb660f service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:06.889276 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91610a-e743-4987-9a97-2c7b4ceb660f service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:06.889670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91610a-e743-4987-9a97-2c7b4ceb660f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:06.891474 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8a91610a-e743-4987-9a97-2c7b4ceb660f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:06.901766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91610a-e743-4987-9a97-2c7b4ceb660f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:06.903246 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91610a-e743-4987-9a97-2c7b4ceb660f service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:06.911648 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1149/2305] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:37:07.254480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02407d-7395-4783-8b3f-bcfc2c29b8f5 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:07.256647 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2a02407d-7395-4783-8b3f-bcfc2c29b8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:07.370981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02407d-7395-4783-8b3f-bcfc2c29b8f5 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:07.370981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ce276-d7a7-4af4-a630-d27e64fc0d42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784b6147de91435292e4cf3940bcad9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:07.370981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ce276-d7a7-4af4-a630-d27e64fc0d42 service placement] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:07.370981 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ce276-d7a7-4af4-a630-d27e64fc0d42 service placement] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:07.371343 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ce276-d7a7-4af4-a630-d27e64fc0d42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:07.372794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-603ce276-d7a7-4af4-a630-d27e64fc0d42 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:07.388039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ce276-d7a7-4af4-a630-d27e64fc0d42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:07.390321 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ce276-d7a7-4af4-a630-d27e64fc0d42 service placement] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:07.406868 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1157/2306] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:09.158781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbdbd0-da65-427f-b5df-9ac59af2aab9 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:09.159616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbdbd0-da65-427f-b5df-9ac59af2aab9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:09.159959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbdbd0-da65-427f-b5df-9ac59af2aab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:09.197746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f8cbdbd0-da65-427f-b5df-9ac59af2aab9 None None] MFA Rules not processed for user `53c0721dc18343b18993ff5af098841d`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:09.199111 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f8cbdbd0-da65-427f-b5df-9ac59af2aab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:09.206624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1150/2307] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:09.284792 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ce276-d7a7-4af4-a630-d27e64fc0d42 service placement] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:09.296455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ce276-d7a7-4af4-a630-d27e64fc0d42 service placement] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:09.301999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608dd891-0bf0-4e81-a779-3a1a6df30637 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784b6147de91435292e4cf3940bcad9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:09.303095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608dd891-0bf0-4e81-a779-3a1a6df30637 service placement] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:09.304187 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608dd891-0bf0-4e81-a779-3a1a6df30637 service placement] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:09.304696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608dd891-0bf0-4e81-a779-3a1a6df30637 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:09.308082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-608dd891-0bf0-4e81-a779-3a1a6df30637 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:09.327052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608dd891-0bf0-4e81-a779-3a1a6df30637 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:09.332946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608dd891-0bf0-4e81-a779-3a1a6df30637 service placement] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:09.347177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1158/2308] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:09.370136 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cbdbd0-da65-427f-b5df-9ac59af2aab9 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:09.371573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f8cbdbd0-da65-427f-b5df-9ac59af2aab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:09.391226 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cbdbd0-da65-427f-b5df-9ac59af2aab9 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:09.397241 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0453996-4fa5-45b1-91d0-498db2819988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:09.398605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0453996-4fa5-45b1-91d0-498db2819988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:09.399310 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0453996-4fa5-45b1-91d0-498db2819988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:09.399725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0453996-4fa5-45b1-91d0-498db2819988 admin admin] PATH_INFO: `/v3/users/67d86f09c6304121be2021cb5c15e63e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:09.402440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0453996-4fa5-45b1-91d0-498db2819988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67d86f09c6304121be2021cb5c15e63e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:09.403388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0453996-4fa5-45b1-91d0-498db2819988 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:09.463154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a0453996-4fa5-45b1-91d0-498db2819988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67d86f09c6304121be2021cb5c15e63e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:09.463824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a0453996-4fa5-45b1-91d0-498db2819988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67d86f09c6304121be2021cb5c15e63e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:09.464615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a0453996-4fa5-45b1-91d0-498db2819988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67d86f09c6304121be2021cb5c15e63e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:09.474298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a0453996-4fa5-45b1-91d0-498db2819988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67d86f09c6304121be2021cb5c15e63e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:09.484581 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:09.491472 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a0453996-4fa5-45b1-91d0-498db2819988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67d86f09c6304121be2021cb5c15e63e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:09.514409 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1151/2309] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:37:09 2022] DELETE /identity/v3/users/67d86f09c6304121be2021cb5c15e63e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:09.907178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608dd891-0bf0-4e81-a779-3a1a6df30637 service placement] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:09.925400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608dd891-0bf0-4e81-a779-3a1a6df30637 service placement] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:10.008941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6375be-861e-49d6-8acc-a78ca6f4d629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:10.009680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6375be-861e-49d6-8acc-a78ca6f4d629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:10.009865 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6375be-861e-49d6-8acc-a78ca6f4d629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:10.010140 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6375be-861e-49d6-8acc-a78ca6f4d629 admin admin] PATH_INFO: `/v3/projects/d21ab544d0b547919794e3efb49ba0e9` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:10.011357 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6375be-861e-49d6-8acc-a78ca6f4d629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d21ab544d0b547919794e3efb49ba0e9)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:10.013291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6375be-861e-49d6-8acc-a78ca6f4d629 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:10.024608 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:10.095386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0d6375be-861e-49d6-8acc-a78ca6f4d629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd21ab544d0b547919794e3efb49ba0e9', 'request_id': 'req-0d6375be-861e-49d6-8acc-a78ca6f4d629'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:10.103662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0d6375be-861e-49d6-8acc-a78ca6f4d629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd21ab544d0b547919794e3efb49ba0e9', 'request_id': 'req-0d6375be-861e-49d6-8acc-a78ca6f4d629'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:10.115142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0d6375be-861e-49d6-8acc-a78ca6f4d629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd21ab544d0b547919794e3efb49ba0e9', 'request_id': 'req-0d6375be-861e-49d6-8acc-a78ca6f4d629'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:10.136559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0d6375be-861e-49d6-8acc-a78ca6f4d629 admin admin] The token cache is being invalidate because project d21ab544d0b547919794e3efb49ba0e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:37:10.137109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0d6375be-861e-49d6-8acc-a78ca6f4d629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:10.137944 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0d6375be-861e-49d6-8acc-a78ca6f4d629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:10.140507 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1159/2310] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:37:09 2022] DELETE /identity/v3/projects/d21ab544d0b547919794e3efb49ba0e9 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:10.145512 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1effa-f602-481e-890f-a427981dcb53 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:10.145879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1effa-f602-481e-890f-a427981dcb53 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:10.146183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1effa-f602-481e-890f-a427981dcb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:10.174849 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-99f1effa-f602-481e-890f-a427981dcb53 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:10.176161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-99f1effa-f602-481e-890f-a427981dcb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:10.246858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1152/2311] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:10.254345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d342-b180-44f3-9673-c9eec48810ef None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:10.254664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d342-b180-44f3-9673-c9eec48810ef None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:10.254940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d342-b180-44f3-9673-c9eec48810ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:10.277128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d676d342-b180-44f3-9673-c9eec48810ef None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:10.278581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d676d342-b180-44f3-9673-c9eec48810ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:10.290342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1160/2312] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:10.294791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1effa-f602-481e-890f-a427981dcb53 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:10.305558 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1effa-f602-481e-890f-a427981dcb53 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:10.310635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424336f-b924-42e7-9bf2-46b3e540285c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:10.312080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424336f-b924-42e7-9bf2-46b3e540285c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:10.312578 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424336f-b924-42e7-9bf2-46b3e540285c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:10.313023 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424336f-b924-42e7-9bf2-46b3e540285c admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:10.314303 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424336f-b924-42e7-9bf2-46b3e540285c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:37:10.314884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424336f-b924-42e7-9bf2-46b3e540285c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:10.318037 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424336f-b924-42e7-9bf2-46b3e540285c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:10.336248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1153/2313] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:37:10.340785 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676d342-b180-44f3-9673-c9eec48810ef None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:10.352490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676d342-b180-44f3-9673-c9eec48810ef None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:10.367875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36598b56-6c6a-40f8-b1ed-cc3f2be7cd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:10.368530 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36598b56-6c6a-40f8-b1ed-cc3f2be7cd2f admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:10.368959 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36598b56-6c6a-40f8-b1ed-cc3f2be7cd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:10.369377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36598b56-6c6a-40f8-b1ed-cc3f2be7cd2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:10.370077 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36598b56-6c6a-40f8-b1ed-cc3f2be7cd2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:10.372061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36598b56-6c6a-40f8-b1ed-cc3f2be7cd2f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:10.402115 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1161/2314] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:37:10 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:37:10.407032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424336f-b924-42e7-9bf2-46b3e540285c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:10.421164 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424336f-b924-42e7-9bf2-46b3e540285c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:10.482330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1cb75-d65e-44a1-ac57-5379de4671a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:10.483132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1cb75-d65e-44a1-ac57-5379de4671a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:10.483427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1cb75-d65e-44a1-ac57-5379de4671a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:10.483691 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1cb75-d65e-44a1-ac57-5379de4671a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:10.484300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1cb75-d65e-44a1-ac57-5379de4671a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:10.486220 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1cb75-d65e-44a1-ac57-5379de4671a3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:10.513470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1154/2315] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:37:10 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:37:10.518040 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36598b56-6c6a-40f8-b1ed-cc3f2be7cd2f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:10.528838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36598b56-6c6a-40f8-b1ed-cc3f2be7cd2f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:10.534846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab207c14-485d-4baa-b5e2-12f78ddec101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:10.535632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab207c14-485d-4baa-b5e2-12f78ddec101 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:10.536054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab207c14-485d-4baa-b5e2-12f78ddec101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:10.536619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab207c14-485d-4baa-b5e2-12f78ddec101 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:10.537412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab207c14-485d-4baa-b5e2-12f78ddec101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:10.539208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab207c14-485d-4baa-b5e2-12f78ddec101 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:10.550328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1162/2316] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:10.554591 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1cb75-d65e-44a1-ac57-5379de4671a3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:10.565353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1cb75-d65e-44a1-ac57-5379de4671a3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:10.571098 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83471be8-d818-4ca2-8098-0c42fbff6056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:10.571375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83471be8-d818-4ca2-8098-0c42fbff6056 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:10.571650 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83471be8-d818-4ca2-8098-0c42fbff6056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:10.571917 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83471be8-d818-4ca2-8098-0c42fbff6056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:10.572618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83471be8-d818-4ca2-8098-0c42fbff6056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:10.574252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83471be8-d818-4ca2-8098-0c42fbff6056 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:10.590986 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1155/2317] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:10.596227 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab207c14-485d-4baa-b5e2-12f78ddec101 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:10.621008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab207c14-485d-4baa-b5e2-12f78ddec101 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:10.628420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d39bf7-8d20-4ca2-aca9-7a0fe663a3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:10.629141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d39bf7-8d20-4ca2-aca9-7a0fe663a3b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:10.629691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d39bf7-8d20-4ca2-aca9-7a0fe663a3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:10.630070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d39bf7-8d20-4ca2-aca9-7a0fe663a3b6 admin admin] PATH_INFO: `/v3/projects/eaf3fd2238b04a1fafd2c6e7055edeb2/users/fadb20b58d984c0fb470661662c95da8/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:10.656977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d39bf7-8d20-4ca2-aca9-7a0fe663a3b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf3fd2238b04a1fafd2c6e7055edeb2, user_id=fadb20b58d984c0fb470661662c95da8, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:10.659597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d39bf7-8d20-4ca2-aca9-7a0fe663a3b6 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:10.684609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1163/2318] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:37:10 2022] PUT /identity/v3/projects/eaf3fd2238b04a1fafd2c6e7055edeb2/users/fadb20b58d984c0fb470661662c95da8/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:10.697778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d6e82-d424-42a3-bcf8-66562949d66c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:10.697990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d6e82-d424-42a3-bcf8-66562949d66c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:10.698549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d6e82-d424-42a3-bcf8-66562949d66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:10.732748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-6a3d6e82-d424-42a3-bcf8-66562949d66c None None] MFA Rules not processed for user `fadb20b58d984c0fb470661662c95da8`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:10.734439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6a3d6e82-d424-42a3-bcf8-66562949d66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:10.787611 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:10.843630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1156/2319] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:10.857791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d39bf7-8d20-4ca2-aca9-7a0fe663a3b6 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:10.859677 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a5d39bf7-8d20-4ca2-aca9-7a0fe663a3b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:10.949109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d39bf7-8d20-4ca2-aca9-7a0fe663a3b6 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:10.955659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119d83a-5dd9-4eb0-92d6-d80825b5d922 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:10.956506 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119d83a-5dd9-4eb0-92d6-d80825b5d922 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:10.956778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119d83a-5dd9-4eb0-92d6-d80825b5d922 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:10.957215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119d83a-5dd9-4eb0-92d6-d80825b5d922 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:10.965589 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119d83a-5dd9-4eb0-92d6-d80825b5d922 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:10.967221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119d83a-5dd9-4eb0-92d6-d80825b5d922 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:10.976170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1164/2320] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:13.727601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d6e82-d424-42a3-bcf8-66562949d66c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:13.792218 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d6e82-d424-42a3-bcf8-66562949d66c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:13.797039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97186ce3-bbc8-427d-9909-094f98dd3e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:13.797666 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97186ce3-bbc8-427d-9909-094f98dd3e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:13.797942 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97186ce3-bbc8-427d-9909-094f98dd3e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:13.798205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97186ce3-bbc8-427d-9909-094f98dd3e54 admin admin] PATH_INFO: `/v3/users/fadb20b58d984c0fb470661662c95da8` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:13.799389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97186ce3-bbc8-427d-9909-094f98dd3e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fadb20b58d984c0fb470661662c95da8)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:13.801345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97186ce3-bbc8-427d-9909-094f98dd3e54 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:13.857752 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-97186ce3-bbc8-427d-9909-094f98dd3e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fadb20b58d984c0fb470661662c95da8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:13.858389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-97186ce3-bbc8-427d-9909-094f98dd3e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fadb20b58d984c0fb470661662c95da8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:13.859012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-97186ce3-bbc8-427d-9909-094f98dd3e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fadb20b58d984c0fb470661662c95da8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:13.866354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-97186ce3-bbc8-427d-9909-094f98dd3e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fadb20b58d984c0fb470661662c95da8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:13.879915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-97186ce3-bbc8-427d-9909-094f98dd3e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fadb20b58d984c0fb470661662c95da8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:13.893756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1157/2321] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:37:13 2022] DELETE /identity/v3/users/fadb20b58d984c0fb470661662c95da8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:13.898828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd285a-82cc-4d5e-b322-f536921c5767 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:13.899326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd285a-82cc-4d5e-b322-f536921c5767 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:13.899418 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd285a-82cc-4d5e-b322-f536921c5767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:13.920029 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-a7bd285a-82cc-4d5e-b322-f536921c5767 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:13.921246 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a7bd285a-82cc-4d5e-b322-f536921c5767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:13.935751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:13.989267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1165/2322] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:13.999280 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97186ce3-bbc8-427d-9909-094f98dd3e54 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:14.000779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-97186ce3-bbc8-427d-9909-094f98dd3e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:14.059288 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97186ce3-bbc8-427d-9909-094f98dd3e54 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:14.062980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c84b4-e405-4c06-8ae2-bd975ab0c1bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:14.063863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c84b4-e405-4c06-8ae2-bd975ab0c1bb service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:14.064179 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c84b4-e405-4c06-8ae2-bd975ab0c1bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:14.064449 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c84b4-e405-4c06-8ae2-bd975ab0c1bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:14.075083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c84b4-e405-4c06-8ae2-bd975ab0c1bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:14.076539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c84b4-e405-4c06-8ae2-bd975ab0c1bb service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:14.087949 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1158/2323] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:14.364701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd285a-82cc-4d5e-b322-f536921c5767 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:14.366199 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a7bd285a-82cc-4d5e-b322-f536921c5767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:14.386816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd285a-82cc-4d5e-b322-f536921c5767 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:14.407167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a48826-8fc0-4de3-bf74-771dc31b080d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:14.407950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a48826-8fc0-4de3-bf74-771dc31b080d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:14.408318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a48826-8fc0-4de3-bf74-771dc31b080d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:14.408678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a48826-8fc0-4de3-bf74-771dc31b080d admin admin] PATH_INFO: `/v3/projects/eaf3fd2238b04a1fafd2c6e7055edeb2` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:14.409924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a48826-8fc0-4de3-bf74-771dc31b080d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf3fd2238b04a1fafd2c6e7055edeb2)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:14.412033 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a48826-8fc0-4de3-bf74-771dc31b080d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:14.462995 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e8a48826-8fc0-4de3-bf74-771dc31b080d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf3fd2238b04a1fafd2c6e7055edeb2', 'request_id': 'req-e8a48826-8fc0-4de3-bf74-771dc31b080d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:14.469179 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e8a48826-8fc0-4de3-bf74-771dc31b080d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf3fd2238b04a1fafd2c6e7055edeb2', 'request_id': 'req-e8a48826-8fc0-4de3-bf74-771dc31b080d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:14.476884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e8a48826-8fc0-4de3-bf74-771dc31b080d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf3fd2238b04a1fafd2c6e7055edeb2', 'request_id': 'req-e8a48826-8fc0-4de3-bf74-771dc31b080d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:14.487485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e8a48826-8fc0-4de3-bf74-771dc31b080d admin admin] The token cache is being invalidate because project eaf3fd2238b04a1fafd2c6e7055edeb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:37:14.487792 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e8a48826-8fc0-4de3-bf74-771dc31b080d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:14.488390 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e8a48826-8fc0-4de3-bf74-771dc31b080d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:14.489910 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1166/2324] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:37:14 2022] DELETE /identity/v3/projects/eaf3fd2238b04a1fafd2c6e7055edeb2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:14.494820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66306fc2-a1aa-4a5d-a65c-165f373b2db5 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:14.495207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66306fc2-a1aa-4a5d-a65c-165f373b2db5 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:14.495371 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66306fc2-a1aa-4a5d-a65c-165f373b2db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:14.503049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:14.527910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-66306fc2-a1aa-4a5d-a65c-165f373b2db5 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:14.529119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-66306fc2-a1aa-4a5d-a65c-165f373b2db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:14.588971 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1159/2325] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:14.596918 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708b2e1-a257-4d8d-af7b-e6891f59f42e None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:14.597201 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708b2e1-a257-4d8d-af7b-e6891f59f42e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:14.597470 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708b2e1-a257-4d8d-af7b-e6891f59f42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:14.617608 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f708b2e1-a257-4d8d-af7b-e6891f59f42e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:14.618828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f708b2e1-a257-4d8d-af7b-e6891f59f42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:14.628211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1167/2326] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:37:14.632865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66306fc2-a1aa-4a5d-a65c-165f373b2db5 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:14.644735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66306fc2-a1aa-4a5d-a65c-165f373b2db5 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:14.649912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6bc64-9fbc-4270-a0a3-0099235cb209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:14.650531 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6bc64-9fbc-4270-a0a3-0099235cb209 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:14.650846 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6bc64-9fbc-4270-a0a3-0099235cb209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:14.651115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6bc64-9fbc-4270-a0a3-0099235cb209 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:14.651762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6bc64-9fbc-4270-a0a3-0099235cb209 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:37:14.652065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6bc64-9fbc-4270-a0a3-0099235cb209 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:14.653679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6bc64-9fbc-4270-a0a3-0099235cb209 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:14.666757 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1160/2327] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:37:14.671323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708b2e1-a257-4d8d-af7b-e6891f59f42e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:14.681567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708b2e1-a257-4d8d-af7b-e6891f59f42e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:14.687613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b12413-908a-447e-9a7f-c58ddd23a9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:14.688210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b12413-908a-447e-9a7f-c58ddd23a9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:14.688485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b12413-908a-447e-9a7f-c58ddd23a9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:14.688751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b12413-908a-447e-9a7f-c58ddd23a9a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:14.689376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b12413-908a-447e-9a7f-c58ddd23a9a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:14.691381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b12413-908a-447e-9a7f-c58ddd23a9a6 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:14.721812 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1168/2328] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:37:14 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:37:14.726054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6bc64-9fbc-4270-a0a3-0099235cb209 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:14.736561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6bc64-9fbc-4270-a0a3-0099235cb209 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:14.796643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a60637-60bd-4b64-8ef2-f9158acfe913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:14.797240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a60637-60bd-4b64-8ef2-f9158acfe913 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:14.797513 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a60637-60bd-4b64-8ef2-f9158acfe913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:14.797775 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a60637-60bd-4b64-8ef2-f9158acfe913 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:14.798384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a60637-60bd-4b64-8ef2-f9158acfe913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:14.800356 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a60637-60bd-4b64-8ef2-f9158acfe913 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:14.823205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1161/2329] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:37:14 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:37:14.827666 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b12413-908a-447e-9a7f-c58ddd23a9a6 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:14.838522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b12413-908a-447e-9a7f-c58ddd23a9a6 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:14.844718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a11776-3141-4e11-ab12-af6d2f473b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:14.845393 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a11776-3141-4e11-ab12-af6d2f473b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:14.845663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a11776-3141-4e11-ab12-af6d2f473b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:14.846024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a11776-3141-4e11-ab12-af6d2f473b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:14.846814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a11776-3141-4e11-ab12-af6d2f473b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:14.848456 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a11776-3141-4e11-ab12-af6d2f473b0d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:14.860908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1169/2330] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:14.864897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a60637-60bd-4b64-8ef2-f9158acfe913 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:14.876448 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a60637-60bd-4b64-8ef2-f9158acfe913 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:14.883049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ab5d3-f733-4c0b-9b03-4e9c2966c92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:14.883667 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ab5d3-f733-4c0b-9b03-4e9c2966c92b admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:14.886711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ab5d3-f733-4c0b-9b03-4e9c2966c92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:14.887041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ab5d3-f733-4c0b-9b03-4e9c2966c92b admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:14.887768 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ab5d3-f733-4c0b-9b03-4e9c2966c92b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:14.889445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ab5d3-f733-4c0b-9b03-4e9c2966c92b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:14.900762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1162/2331] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:14.904995 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a11776-3141-4e11-ab12-af6d2f473b0d admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:14.917619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a11776-3141-4e11-ab12-af6d2f473b0d admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:14.924891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cd9bd-e722-4c1e-814a-da931f82438f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:14.926585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd9bd-e722-4c1e-814a-da931f82438f admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:14.927022 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd9bd-e722-4c1e-814a-da931f82438f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:14.927401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd9bd-e722-4c1e-814a-da931f82438f admin admin] PATH_INFO: `/v3/projects/1b940044fa014a5ab4d5210c38909f74/users/607950bd63cc4e5eb6eadfa2203d8a04/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:14.948742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cd9bd-e722-4c1e-814a-da931f82438f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b940044fa014a5ab4d5210c38909f74, user_id=607950bd63cc4e5eb6eadfa2203d8a04, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:14.952521 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cd9bd-e722-4c1e-814a-da931f82438f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:14.969858 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1170/2332] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:37:14 2022] PUT /identity/v3/projects/1b940044fa014a5ab4d5210c38909f74/users/607950bd63cc4e5eb6eadfa2203d8a04/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:37:14.980298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828806a-48f1-4a28-8469-5bd93b5e1db3 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:14.980909 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828806a-48f1-4a28-8469-5bd93b5e1db3 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:14.981273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828806a-48f1-4a28-8469-5bd93b5e1db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:15.021296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1828806a-48f1-4a28-8469-5bd93b5e1db3 None None] MFA Rules not processed for user `607950bd63cc4e5eb6eadfa2203d8a04`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:15.022556 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1828806a-48f1-4a28-8469-5bd93b5e1db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:15.102702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:15.112354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1163/2333] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:15.115994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cd9bd-e722-4c1e-814a-da931f82438f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:15.178772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:15.200197 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cd9bd-e722-4c1e-814a-da931f82438f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:15.204948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876b06c-4128-4cf4-8877-695354d279d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:15.205557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876b06c-4128-4cf4-8877-695354d279d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:15.205832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876b06c-4128-4cf4-8877-695354d279d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:15.206097 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876b06c-4128-4cf4-8877-695354d279d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:15.206753 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876b06c-4128-4cf4-8877-695354d279d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:15.208982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876b06c-4128-4cf4-8877-695354d279d6 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:15.241770 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1171/2334] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:37:15 2022] POST /identity/v3/projects => generated 363 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:37:15.242621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828806a-48f1-4a28-8469-5bd93b5e1db3 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:15.253726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828806a-48f1-4a28-8469-5bd93b5e1db3 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:15.314150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c36fff-0dfc-4dae-aa04-2649201eb994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:15.314799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c36fff-0dfc-4dae-aa04-2649201eb994 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:15.315079 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c36fff-0dfc-4dae-aa04-2649201eb994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:15.315360 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c36fff-0dfc-4dae-aa04-2649201eb994 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:15.315970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c36fff-0dfc-4dae-aa04-2649201eb994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:15.317993 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c36fff-0dfc-4dae-aa04-2649201eb994 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:15.342535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1164/2335] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:37:15 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:37:15.346225 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876b06c-4128-4cf4-8877-695354d279d6 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:15.360476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876b06c-4128-4cf4-8877-695354d279d6 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:15.368039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001e4a1-8409-4654-b975-860440b96bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:15.368661 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001e4a1-8409-4654-b975-860440b96bfd admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:15.368941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001e4a1-8409-4654-b975-860440b96bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:15.369287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001e4a1-8409-4654-b975-860440b96bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:15.369915 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001e4a1-8409-4654-b975-860440b96bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:15.371601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001e4a1-8409-4654-b975-860440b96bfd admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:15.383331 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1172/2336] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:15.387010 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c36fff-0dfc-4dae-aa04-2649201eb994 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:15.398380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c36fff-0dfc-4dae-aa04-2649201eb994 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:15.404577 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdbe37f-a606-43d2-91ac-60bd3f8da55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:15.405487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbe37f-a606-43d2-91ac-60bd3f8da55d admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:15.405989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbe37f-a606-43d2-91ac-60bd3f8da55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:15.406234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbe37f-a606-43d2-91ac-60bd3f8da55d admin admin] PATH_INFO: `/v3/projects/3713f410ccbc4bbcb1526153cc92ddff/users/2cb6c024391c45259d8b89ed6dfda611/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:15.424306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdbe37f-a606-43d2-91ac-60bd3f8da55d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3713f410ccbc4bbcb1526153cc92ddff, user_id=2cb6c024391c45259d8b89ed6dfda611, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:15.427096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdbe37f-a606-43d2-91ac-60bd3f8da55d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:15.443515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1165/2337] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:37:15 2022] PUT /identity/v3/projects/3713f410ccbc4bbcb1526153cc92ddff/users/2cb6c024391c45259d8b89ed6dfda611/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:37:15.447618 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001e4a1-8409-4654-b975-860440b96bfd admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:15.458378 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001e4a1-8409-4654-b975-860440b96bfd admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:15.513493 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeff781-0360-4654-96a1-cd412ca595b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:15.514101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff781-0360-4654-96a1-cd412ca595b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:15.514419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff781-0360-4654-96a1-cd412ca595b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:15.514766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff781-0360-4654-96a1-cd412ca595b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:15.515457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeff781-0360-4654-96a1-cd412ca595b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:15.517049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeff781-0360-4654-96a1-cd412ca595b6 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:15.527061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1173/2338] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:15.531657 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdbe37f-a606-43d2-91ac-60bd3f8da55d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:15.545633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdbe37f-a606-43d2-91ac-60bd3f8da55d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:15.551293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3040494-ff0f-40c7-8556-1afcb36ffe31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:15.552172 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3040494-ff0f-40c7-8556-1afcb36ffe31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:15.552497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3040494-ff0f-40c7-8556-1afcb36ffe31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:15.552773 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3040494-ff0f-40c7-8556-1afcb36ffe31 admin admin] PATH_INFO: `/v3/projects/3713f410ccbc4bbcb1526153cc92ddff/users/2cb6c024391c45259d8b89ed6dfda611/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:15.554882 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3040494-ff0f-40c7-8556-1afcb36ffe31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3713f410ccbc4bbcb1526153cc92ddff, user_id=2cb6c024391c45259d8b89ed6dfda611, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:15.557525 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3040494-ff0f-40c7-8556-1afcb36ffe31 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:15.569384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1166/2339] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:37:15 2022] PUT /identity/v3/projects/3713f410ccbc4bbcb1526153cc92ddff/users/2cb6c024391c45259d8b89ed6dfda611/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:15.581456 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb66a6-ebe5-42f3-92c8-58a63586d508 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:15.581688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb66a6-ebe5-42f3-92c8-58a63586d508 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:15.581989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb66a6-ebe5-42f3-92c8-58a63586d508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:15.610173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-4afb66a6-ebe5-42f3-92c8-58a63586d508 None None] MFA Rules not processed for user `2cb6c024391c45259d8b89ed6dfda611`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:15.612355 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-4afb66a6-ebe5-42f3-92c8-58a63586d508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:15.686290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1174/2340] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:37:15.698093 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3040494-ff0f-40c7-8556-1afcb36ffe31 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:15.699848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f3040494-ff0f-40c7-8556-1afcb36ffe31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:15.757062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3040494-ff0f-40c7-8556-1afcb36ffe31 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:15.761313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebcbdec-6739-4fbe-bf60-2e0122f3c25d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:15.761913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcbdec-6739-4fbe-bf60-2e0122f3c25d service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:15.762204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcbdec-6739-4fbe-bf60-2e0122f3c25d service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:15.762467 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcbdec-6739-4fbe-bf60-2e0122f3c25d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:15.772172 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebcbdec-6739-4fbe-bf60-2e0122f3c25d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:15.773606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebcbdec-6739-4fbe-bf60-2e0122f3c25d service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:15.783071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1167/2341] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:16.230478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb66a6-ebe5-42f3-92c8-58a63586d508 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:16.272592 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:16.298581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb66a6-ebe5-42f3-92c8-58a63586d508 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:16.303591 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc290b-6358-47d2-9ed3-5404888f5b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:16.304230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc290b-6358-47d2-9ed3-5404888f5b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:16.304506 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc290b-6358-47d2-9ed3-5404888f5b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:16.304772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc290b-6358-47d2-9ed3-5404888f5b85 admin admin] PATH_INFO: `/v3/users/607950bd63cc4e5eb6eadfa2203d8a04` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:16.306001 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc290b-6358-47d2-9ed3-5404888f5b85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=607950bd63cc4e5eb6eadfa2203d8a04)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:16.307976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc290b-6358-47d2-9ed3-5404888f5b85 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:16.352975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-7dbc290b-6358-47d2-9ed3-5404888f5b85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '607950bd63cc4e5eb6eadfa2203d8a04'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:16.353722 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-7dbc290b-6358-47d2-9ed3-5404888f5b85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '607950bd63cc4e5eb6eadfa2203d8a04'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:16.354454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-7dbc290b-6358-47d2-9ed3-5404888f5b85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '607950bd63cc4e5eb6eadfa2203d8a04'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:16.361924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-7dbc290b-6358-47d2-9ed3-5404888f5b85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '607950bd63cc4e5eb6eadfa2203d8a04'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:16.373510 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-7dbc290b-6358-47d2-9ed3-5404888f5b85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '607950bd63cc4e5eb6eadfa2203d8a04'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:16.387397 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1175/2342] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:37:16 2022] DELETE /identity/v3/users/607950bd63cc4e5eb6eadfa2203d8a04 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:16.392010 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c7a1a-9f5b-48be-8a6b-7309163f2861 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:16.392334 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c7a1a-9f5b-48be-8a6b-7309163f2861 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:16.392602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c7a1a-9f5b-48be-8a6b-7309163f2861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:16.411419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-147c7a1a-9f5b-48be-8a6b-7309163f2861 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:16.412604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-147c7a1a-9f5b-48be-8a6b-7309163f2861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:16.458851 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:16.474871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1168/2343] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:37:16.484715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc290b-6358-47d2-9ed3-5404888f5b85 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:16.486282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7dbc290b-6358-47d2-9ed3-5404888f5b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:16.548649 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc290b-6358-47d2-9ed3-5404888f5b85 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:16.552410 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd7c51-9900-4843-a739-6ce509f54c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:16.553024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd7c51-9900-4843-a739-6ce509f54c71 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:16.553300 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd7c51-9900-4843-a739-6ce509f54c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:16.553570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd7c51-9900-4843-a739-6ce509f54c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:16.563517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd7c51-9900-4843-a739-6ce509f54c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:16.565122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd7c51-9900-4843-a739-6ce509f54c71 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:16.575107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1176/2344] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:16.855399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c7a1a-9f5b-48be-8a6b-7309163f2861 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:16.856925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-147c7a1a-9f5b-48be-8a6b-7309163f2861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:16.868713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c7a1a-9f5b-48be-8a6b-7309163f2861 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:16.873363 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:16.873989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:16.874266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:16.874573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] PATH_INFO: `/v3/projects/1b940044fa014a5ab4d5210c38909f74` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:16.875699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b940044fa014a5ab4d5210c38909f74)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:16.877596 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:16.928141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b940044fa014a5ab4d5210c38909f74', 'request_id': 'req-90d3420c-a7b3-434e-b835-b39136696dcd'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:16.934590 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b940044fa014a5ab4d5210c38909f74', 'request_id': 'req-90d3420c-a7b3-434e-b835-b39136696dcd'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:16.940402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b940044fa014a5ab4d5210c38909f74', 'request_id': 'req-90d3420c-a7b3-434e-b835-b39136696dcd'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:16.954245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] The token cache is being invalidate because project 1b940044fa014a5ab4d5210c38909f74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:37:16.954555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:16.955174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:16.956983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1169/2345] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:37:16 2022] DELETE /identity/v3/projects/1b940044fa014a5ab4d5210c38909f74 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:16.961041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd7c51-9900-4843-a739-6ce509f54c71 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:16.971613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd7c51-9900-4843-a739-6ce509f54c71 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:17.024665 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:17.031142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef279d0-abd6-46e7-afe5-374ff9bc6ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:17.031938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef279d0-abd6-46e7-afe5-374ff9bc6ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:17.032336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef279d0-abd6-46e7-afe5-374ff9bc6ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:17.032666 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef279d0-abd6-46e7-afe5-374ff9bc6ad8 admin admin] PATH_INFO: `/v3/users/2cb6c024391c45259d8b89ed6dfda611` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:17.033942 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef279d0-abd6-46e7-afe5-374ff9bc6ad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb6c024391c45259d8b89ed6dfda611)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:17.036674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef279d0-abd6-46e7-afe5-374ff9bc6ad8 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:17.097240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0ef279d0-abd6-46e7-afe5-374ff9bc6ad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb6c024391c45259d8b89ed6dfda611'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:17.097885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0ef279d0-abd6-46e7-afe5-374ff9bc6ad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb6c024391c45259d8b89ed6dfda611'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:17.098586 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0ef279d0-abd6-46e7-afe5-374ff9bc6ad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb6c024391c45259d8b89ed6dfda611'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:17.105795 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0ef279d0-abd6-46e7-afe5-374ff9bc6ad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb6c024391c45259d8b89ed6dfda611'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:17.118108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-0ef279d0-abd6-46e7-afe5-374ff9bc6ad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb6c024391c45259d8b89ed6dfda611'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:17.132539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1177/2346] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:37:16 2022] DELETE /identity/v3/users/2cb6c024391c45259d8b89ed6dfda611 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:17.576237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:17.578153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:17.599760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3420c-a7b3-434e-b835-b39136696dcd admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:17.667570 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4185b1-cfe7-4da7-a067-d31e77503877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:17.668206 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4185b1-cfe7-4da7-a067-d31e77503877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:17.668485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4185b1-cfe7-4da7-a067-d31e77503877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:17.668764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4185b1-cfe7-4da7-a067-d31e77503877 admin admin] PATH_INFO: `/v3/projects/3713f410ccbc4bbcb1526153cc92ddff` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:17.669897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4185b1-cfe7-4da7-a067-d31e77503877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3713f410ccbc4bbcb1526153cc92ddff)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:17.671996 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4185b1-cfe7-4da7-a067-d31e77503877 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:17.720279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fe4185b1-cfe7-4da7-a067-d31e77503877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3713f410ccbc4bbcb1526153cc92ddff', 'request_id': 'req-fe4185b1-cfe7-4da7-a067-d31e77503877'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:17.726265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fe4185b1-cfe7-4da7-a067-d31e77503877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3713f410ccbc4bbcb1526153cc92ddff', 'request_id': 'req-fe4185b1-cfe7-4da7-a067-d31e77503877'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:17.731861 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fe4185b1-cfe7-4da7-a067-d31e77503877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3713f410ccbc4bbcb1526153cc92ddff', 'request_id': 'req-fe4185b1-cfe7-4da7-a067-d31e77503877'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:17.746593 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fe4185b1-cfe7-4da7-a067-d31e77503877 admin admin] The token cache is being invalidate because project 3713f410ccbc4bbcb1526153cc92ddff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:37:17.746874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fe4185b1-cfe7-4da7-a067-d31e77503877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:17.747476 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fe4185b1-cfe7-4da7-a067-d31e77503877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:17.748955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1170/2347] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:37:17 2022] DELETE /identity/v3/projects/3713f410ccbc4bbcb1526153cc92ddff => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:17.754145 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca82889-4743-4a5e-8a79-16e4c353d7c1 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:17.754667 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca82889-4743-4a5e-8a79-16e4c353d7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:17.754783 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca82889-4743-4a5e-8a79-16e4c353d7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:17.774755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-fca82889-4743-4a5e-8a79-16e4c353d7c1 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:17.775934 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-fca82889-4743-4a5e-8a79-16e4c353d7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:17.839198 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1178/2348] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:17.847137 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c700fc-b96b-428e-8836-4cb2120960db None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:17.847419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c700fc-b96b-428e-8836-4cb2120960db None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:17.847688 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c700fc-b96b-428e-8836-4cb2120960db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:17.863275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:17.869499 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-27c700fc-b96b-428e-8836-4cb2120960db None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:17.871664 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-27c700fc-b96b-428e-8836-4cb2120960db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:17.880491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1171/2349] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:17.884709 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca82889-4743-4a5e-8a79-16e4c353d7c1 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:17.894888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca82889-4743-4a5e-8a79-16e4c353d7c1 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:17.900034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9190bbf-97b1-4bbe-860d-2a0b26ffabc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:17.900679 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9190bbf-97b1-4bbe-860d-2a0b26ffabc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:17.901068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9190bbf-97b1-4bbe-860d-2a0b26ffabc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:17.901347 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9190bbf-97b1-4bbe-860d-2a0b26ffabc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:17.902012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9190bbf-97b1-4bbe-860d-2a0b26ffabc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:37:17.902317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9190bbf-97b1-4bbe-860d-2a0b26ffabc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:17.903982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9190bbf-97b1-4bbe-860d-2a0b26ffabc1 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:17.916882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1179/2350] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:37:17.921357 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c700fc-b96b-428e-8836-4cb2120960db None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:17.930605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c700fc-b96b-428e-8836-4cb2120960db None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:17.935088 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592058e-d244-4648-ad9c-4113cf667177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:17.935684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592058e-d244-4648-ad9c-4113cf667177 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:17.935958 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592058e-d244-4648-ad9c-4113cf667177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:17.936223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592058e-d244-4648-ad9c-4113cf667177 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:17.936867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592058e-d244-4648-ad9c-4113cf667177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:17.938765 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592058e-d244-4648-ad9c-4113cf667177 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:17.972265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1172/2351] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:37:17 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:37:17.973740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9190bbf-97b1-4bbe-860d-2a0b26ffabc1 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:17.984929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9190bbf-97b1-4bbe-860d-2a0b26ffabc1 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:18.043195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed346d5-5144-48b0-a413-2335475ace5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:18.043892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed346d5-5144-48b0-a413-2335475ace5d admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:18.044244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed346d5-5144-48b0-a413-2335475ace5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:18.044591 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed346d5-5144-48b0-a413-2335475ace5d admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:18.045297 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed346d5-5144-48b0-a413-2335475ace5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:18.047271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed346d5-5144-48b0-a413-2335475ace5d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:18.057817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:18.071686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1180/2352] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:37:17 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:37:18.075894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592058e-d244-4648-ad9c-4113cf667177 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:18.086686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592058e-d244-4648-ad9c-4113cf667177 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:18.091385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386719d-66c0-446f-8ce5-6dd23a0a7fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:18.091990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386719d-66c0-446f-8ce5-6dd23a0a7fea admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:18.092270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386719d-66c0-446f-8ce5-6dd23a0a7fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:18.092541 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386719d-66c0-446f-8ce5-6dd23a0a7fea admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:18.093184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386719d-66c0-446f-8ce5-6dd23a0a7fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:18.094899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386719d-66c0-446f-8ce5-6dd23a0a7fea admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:18.105092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1173/2353] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:18.109637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed346d5-5144-48b0-a413-2335475ace5d admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:18.120589 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed346d5-5144-48b0-a413-2335475ace5d admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:18.125563 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c6155-5423-4760-8ab0-b3c22a88426d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:18.126160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c6155-5423-4760-8ab0-b3c22a88426d admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:18.126461 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c6155-5423-4760-8ab0-b3c22a88426d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:18.126764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c6155-5423-4760-8ab0-b3c22a88426d admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:18.127408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c6155-5423-4760-8ab0-b3c22a88426d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:18.129042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c6155-5423-4760-8ab0-b3c22a88426d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:18.140144 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1181/2354] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:18.144349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386719d-66c0-446f-8ce5-6dd23a0a7fea admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:18.154717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386719d-66c0-446f-8ce5-6dd23a0a7fea admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:18.159868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11f9ef-8c3b-45ac-a9d3-241fb3c99534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:18.160515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11f9ef-8c3b-45ac-a9d3-241fb3c99534 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:18.160796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11f9ef-8c3b-45ac-a9d3-241fb3c99534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:18.161064 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11f9ef-8c3b-45ac-a9d3-241fb3c99534 admin admin] PATH_INFO: `/v3/projects/58cb67c6da5749ea9b99c6f707e7a3e1/users/34a8d28d90734b178d5097dbf768a706/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:18.176930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11f9ef-8c3b-45ac-a9d3-241fb3c99534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58cb67c6da5749ea9b99c6f707e7a3e1, user_id=34a8d28d90734b178d5097dbf768a706, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:18.179676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11f9ef-8c3b-45ac-a9d3-241fb3c99534 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:18.195089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1174/2355] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:37:18 2022] PUT /identity/v3/projects/58cb67c6da5749ea9b99c6f707e7a3e1/users/34a8d28d90734b178d5097dbf768a706/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:18.205373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160e91-a720-4f61-8a44-c0495195ee6a None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:18.205652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160e91-a720-4f61-8a44-c0495195ee6a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:18.205917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160e91-a720-4f61-8a44-c0495195ee6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:18.234858 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-09160e91-a720-4f61-8a44-c0495195ee6a None None] MFA Rules not processed for user `34a8d28d90734b178d5097dbf768a706`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:18.236034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-09160e91-a720-4f61-8a44-c0495195ee6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:18.302637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1182/2356] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:18.307366 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11f9ef-8c3b-45ac-a9d3-241fb3c99534 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:18.371213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11f9ef-8c3b-45ac-a9d3-241fb3c99534 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:18.376345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38fcef-4e83-4877-9ccb-c1fab98620e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:18.376948 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38fcef-4e83-4877-9ccb-c1fab98620e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:18.377219 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38fcef-4e83-4877-9ccb-c1fab98620e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:18.377485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38fcef-4e83-4877-9ccb-c1fab98620e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:18.378095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38fcef-4e83-4877-9ccb-c1fab98620e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:18.380126 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38fcef-4e83-4877-9ccb-c1fab98620e2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:18.411308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1175/2357] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:37:18 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:37:18.416517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09160e91-a720-4f61-8a44-c0495195ee6a None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:18.430996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09160e91-a720-4f61-8a44-c0495195ee6a None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:18.487141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379825c-a0c5-4b68-b2b9-e151046e995a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:18.487671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379825c-a0c5-4b68-b2b9-e151046e995a admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:18.487951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379825c-a0c5-4b68-b2b9-e151046e995a admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:18.488223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379825c-a0c5-4b68-b2b9-e151046e995a admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:18.488850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379825c-a0c5-4b68-b2b9-e151046e995a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:18.490874 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379825c-a0c5-4b68-b2b9-e151046e995a admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:18.518588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1183/2358] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:37:18 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:37:18.519015 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38fcef-4e83-4877-9ccb-c1fab98620e2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:18.529846 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38fcef-4e83-4877-9ccb-c1fab98620e2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:18.535008 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca278b1-2acc-4358-9ebb-a881df2f372a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:18.535627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca278b1-2acc-4358-9ebb-a881df2f372a admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:18.536059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca278b1-2acc-4358-9ebb-a881df2f372a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:18.536332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca278b1-2acc-4358-9ebb-a881df2f372a admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:18.537117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca278b1-2acc-4358-9ebb-a881df2f372a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:18.539052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca278b1-2acc-4358-9ebb-a881df2f372a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:18.550308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1176/2359] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:18.555876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379825c-a0c5-4b68-b2b9-e151046e995a admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:18.566181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379825c-a0c5-4b68-b2b9-e151046e995a admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:18.571887 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eaec0c-319e-46f4-96b5-c0ee61779c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:18.572539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eaec0c-319e-46f4-96b5-c0ee61779c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:18.572797 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eaec0c-319e-46f4-96b5-c0ee61779c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:18.573065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eaec0c-319e-46f4-96b5-c0ee61779c54 admin admin] PATH_INFO: `/v3/projects/7b1587bc173940b5ae61bedf88c9d878/users/c1a1cfb797ec4797b9f1b3d00d5d2f5d/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:18.587722 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eaec0c-319e-46f4-96b5-c0ee61779c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1587bc173940b5ae61bedf88c9d878, user_id=c1a1cfb797ec4797b9f1b3d00d5d2f5d, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:18.590389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eaec0c-319e-46f4-96b5-c0ee61779c54 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:18.605422 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1184/2360] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:37:18 2022] PUT /identity/v3/projects/7b1587bc173940b5ae61bedf88c9d878/users/c1a1cfb797ec4797b9f1b3d00d5d2f5d/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:18.609738 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca278b1-2acc-4358-9ebb-a881df2f372a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:18.629916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca278b1-2acc-4358-9ebb-a881df2f372a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:18.696827 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865c02d-2875-4639-ae28-ba06cbd313ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:18.697450 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865c02d-2875-4639-ae28-ba06cbd313ad admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:18.697728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865c02d-2875-4639-ae28-ba06cbd313ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:18.697989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865c02d-2875-4639-ae28-ba06cbd313ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:18.698674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865c02d-2875-4639-ae28-ba06cbd313ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:18.700279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865c02d-2875-4639-ae28-ba06cbd313ad admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:18.710229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1177/2361] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:18.714641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eaec0c-319e-46f4-96b5-c0ee61779c54 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:18.725334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eaec0c-319e-46f4-96b5-c0ee61779c54 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:18.730413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60158ad0-1260-4ad1-82a5-9c7aa80c57c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:18.731125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60158ad0-1260-4ad1-82a5-9c7aa80c57c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:18.731499 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60158ad0-1260-4ad1-82a5-9c7aa80c57c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:18.731921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60158ad0-1260-4ad1-82a5-9c7aa80c57c1 admin admin] PATH_INFO: `/v3/projects/7b1587bc173940b5ae61bedf88c9d878/users/c1a1cfb797ec4797b9f1b3d00d5d2f5d/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:18.734055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60158ad0-1260-4ad1-82a5-9c7aa80c57c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1587bc173940b5ae61bedf88c9d878, user_id=c1a1cfb797ec4797b9f1b3d00d5d2f5d, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:18.736791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60158ad0-1260-4ad1-82a5-9c7aa80c57c1 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:18.748077 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1185/2362] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:37:18 2022] PUT /identity/v3/projects/7b1587bc173940b5ae61bedf88c9d878/users/c1a1cfb797ec4797b9f1b3d00d5d2f5d/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:37:18.759880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be348b6-4ad3-488e-85b6-7fe993f65821 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:18.760154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be348b6-4ad3-488e-85b6-7fe993f65821 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:18.760464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be348b6-4ad3-488e-85b6-7fe993f65821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:18.786794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:18.790046 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7be348b6-4ad3-488e-85b6-7fe993f65821 None None] MFA Rules not processed for user `c1a1cfb797ec4797b9f1b3d00d5d2f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:18.792604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7be348b6-4ad3-488e-85b6-7fe993f65821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:18.862198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1178/2363] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:18.873486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60158ad0-1260-4ad1-82a5-9c7aa80c57c1 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:18.875584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-60158ad0-1260-4ad1-82a5-9c7aa80c57c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:18.905272 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:18.940975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60158ad0-1260-4ad1-82a5-9c7aa80c57c1 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:18.945068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22258ddb-6d46-4195-aaa0-3147479ebb0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:18.945690 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22258ddb-6d46-4195-aaa0-3147479ebb0f service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:18.946210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22258ddb-6d46-4195-aaa0-3147479ebb0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:18.946489 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22258ddb-6d46-4195-aaa0-3147479ebb0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:18.956472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22258ddb-6d46-4195-aaa0-3147479ebb0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:18.958016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22258ddb-6d46-4195-aaa0-3147479ebb0f service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:18.969302 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1186/2364] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:20.454709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be348b6-4ad3-488e-85b6-7fe993f65821 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:20.521091 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be348b6-4ad3-488e-85b6-7fe993f65821 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:20.525837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674f366-3c28-4429-8e5a-495f77b54a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:20.526516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674f366-3c28-4429-8e5a-495f77b54a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:20.526801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674f366-3c28-4429-8e5a-495f77b54a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:20.527413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674f366-3c28-4429-8e5a-495f77b54a70 admin admin] PATH_INFO: `/v3/users/34a8d28d90734b178d5097dbf768a706` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:20.528395 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674f366-3c28-4429-8e5a-495f77b54a70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34a8d28d90734b178d5097dbf768a706)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:20.530736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674f366-3c28-4429-8e5a-495f77b54a70 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:20.581604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6674f366-3c28-4429-8e5a-495f77b54a70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34a8d28d90734b178d5097dbf768a706'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:20.582260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6674f366-3c28-4429-8e5a-495f77b54a70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34a8d28d90734b178d5097dbf768a706'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:20.583129 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6674f366-3c28-4429-8e5a-495f77b54a70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34a8d28d90734b178d5097dbf768a706'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:20.591102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6674f366-3c28-4429-8e5a-495f77b54a70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34a8d28d90734b178d5097dbf768a706'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:20.604290 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6674f366-3c28-4429-8e5a-495f77b54a70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34a8d28d90734b178d5097dbf768a706'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:20.619518 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1179/2365] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:37:20 2022] DELETE /identity/v3/users/34a8d28d90734b178d5097dbf768a706 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:20.624209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ed5f1-c27e-4bc7-a251-45aee0b8632e None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:20.625066 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ed5f1-c27e-4bc7-a251-45aee0b8632e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:20.625356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ed5f1-c27e-4bc7-a251-45aee0b8632e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:20.645722 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-c67ed5f1-c27e-4bc7-a251-45aee0b8632e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:20.646994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c67ed5f1-c27e-4bc7-a251-45aee0b8632e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:20.709289 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1187/2366] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:37:20.719638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674f366-3c28-4429-8e5a-495f77b54a70 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:20.721349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6674f366-3c28-4429-8e5a-495f77b54a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:20.744503 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:20.790017 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674f366-3c28-4429-8e5a-495f77b54a70 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:20.794425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa387c-74cb-4de7-bda3-090566494ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:20.795076 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa387c-74cb-4de7-bda3-090566494ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:20.795353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa387c-74cb-4de7-bda3-090566494ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:20.796044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa387c-74cb-4de7-bda3-090566494ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:20.805860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa387c-74cb-4de7-bda3-090566494ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:20.807044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa387c-74cb-4de7-bda3-090566494ef4 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:20.816781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1180/2367] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:21.102986 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ed5f1-c27e-4bc7-a251-45aee0b8632e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:21.108970 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c67ed5f1-c27e-4bc7-a251-45aee0b8632e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:21.128084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ed5f1-c27e-4bc7-a251-45aee0b8632e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:21.135359 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:21.136121 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:21.136630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:21.136988 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] PATH_INFO: `/v3/projects/58cb67c6da5749ea9b99c6f707e7a3e1` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:21.138792 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] RBAC: Authorizing `identity:delete_project(project_id=58cb67c6da5749ea9b99c6f707e7a3e1)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:21.142379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:21.208545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58cb67c6da5749ea9b99c6f707e7a3e1', 'request_id': 'req-07d8cb17-6de9-429d-bff0-8c9a815bcdde'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.220480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58cb67c6da5749ea9b99c6f707e7a3e1', 'request_id': 'req-07d8cb17-6de9-429d-bff0-8c9a815bcdde'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.233577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58cb67c6da5749ea9b99c6f707e7a3e1', 'request_id': 'req-07d8cb17-6de9-429d-bff0-8c9a815bcdde'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.249680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] The token cache is being invalidate because project 58cb67c6da5749ea9b99c6f707e7a3e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:37:21.250357 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.251332 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.253217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1188/2368] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:37:21 2022] DELETE /identity/v3/projects/58cb67c6da5749ea9b99c6f707e7a3e1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:21.258019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa387c-74cb-4de7-bda3-090566494ef4 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:21.271461 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa387c-74cb-4de7-bda3-090566494ef4 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:21.358960 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68b3f7-ff35-4a8a-93ab-0e383bf416d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:21.359793 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68b3f7-ff35-4a8a-93ab-0e383bf416d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:21.360665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68b3f7-ff35-4a8a-93ab-0e383bf416d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:21.361043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68b3f7-ff35-4a8a-93ab-0e383bf416d5 admin admin] PATH_INFO: `/v3/users/c1a1cfb797ec4797b9f1b3d00d5d2f5d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:21.363042 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68b3f7-ff35-4a8a-93ab-0e383bf416d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1a1cfb797ec4797b9f1b3d00d5d2f5d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:21.365876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68b3f7-ff35-4a8a-93ab-0e383bf416d5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:21.440987 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fc68b3f7-ff35-4a8a-93ab-0e383bf416d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1a1cfb797ec4797b9f1b3d00d5d2f5d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.441798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fc68b3f7-ff35-4a8a-93ab-0e383bf416d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1a1cfb797ec4797b9f1b3d00d5d2f5d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.442529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fc68b3f7-ff35-4a8a-93ab-0e383bf416d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1a1cfb797ec4797b9f1b3d00d5d2f5d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.452321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fc68b3f7-ff35-4a8a-93ab-0e383bf416d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1a1cfb797ec4797b9f1b3d00d5d2f5d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.465021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-fc68b3f7-ff35-4a8a-93ab-0e383bf416d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1a1cfb797ec4797b9f1b3d00d5d2f5d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.483126 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1181/2369] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:37:21 2022] DELETE /identity/v3/users/c1a1cfb797ec4797b9f1b3d00d5d2f5d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:21.784116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:21.785730 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:21.800450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8cb17-6de9-429d-bff0-8c9a815bcdde admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:21.860555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:21.872403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f757bb-d607-4b83-93e9-de2b51213cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:21.873060 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f757bb-d607-4b83-93e9-de2b51213cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:21.873337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f757bb-d607-4b83-93e9-de2b51213cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:21.873606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f757bb-d607-4b83-93e9-de2b51213cc8 admin admin] PATH_INFO: `/v3/projects/7b1587bc173940b5ae61bedf88c9d878` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:21.874951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f757bb-d607-4b83-93e9-de2b51213cc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b1587bc173940b5ae61bedf88c9d878)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:21.876883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f757bb-d607-4b83-93e9-de2b51213cc8 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:21.927534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-62f757bb-d607-4b83-93e9-de2b51213cc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b1587bc173940b5ae61bedf88c9d878', 'request_id': 'req-62f757bb-d607-4b83-93e9-de2b51213cc8'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.933825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-62f757bb-d607-4b83-93e9-de2b51213cc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b1587bc173940b5ae61bedf88c9d878', 'request_id': 'req-62f757bb-d607-4b83-93e9-de2b51213cc8'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.942620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-62f757bb-d607-4b83-93e9-de2b51213cc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b1587bc173940b5ae61bedf88c9d878', 'request_id': 'req-62f757bb-d607-4b83-93e9-de2b51213cc8'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.953319 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-62f757bb-d607-4b83-93e9-de2b51213cc8 admin admin] The token cache is being invalidate because project 7b1587bc173940b5ae61bedf88c9d878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:37:21.953607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-62f757bb-d607-4b83-93e9-de2b51213cc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.954306 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-62f757bb-d607-4b83-93e9-de2b51213cc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:21.955975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1189/2370] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:37:21 2022] DELETE /identity/v3/projects/7b1587bc173940b5ae61bedf88c9d878 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:21.960864 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ec6d1-6e8b-49e9-96a6-fd36f186acff None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:21.961147 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ec6d1-6e8b-49e9-96a6-fd36f186acff None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:21.961418 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ec6d1-6e8b-49e9-96a6-fd36f186acff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:21.981890 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ec6ec6d1-6e8b-49e9-96a6-fd36f186acff None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:21.983119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ec6ec6d1-6e8b-49e9-96a6-fd36f186acff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:22.015649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:22.046750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1182/2371] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:22.054706 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20513f2-fe6d-4009-89d3-4be149f9bd56 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.054992 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20513f2-fe6d-4009-89d3-4be149f9bd56 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.055388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20513f2-fe6d-4009-89d3-4be149f9bd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:22.076360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d20513f2-fe6d-4009-89d3-4be149f9bd56 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:22.077577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d20513f2-fe6d-4009-89d3-4be149f9bd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:22.091079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1190/2372] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:37:22.096420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ec6d1-6e8b-49e9-96a6-fd36f186acff None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:22.111495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ec6d1-6e8b-49e9-96a6-fd36f186acff None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:22.117237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e2715-7d2c-4f8c-88e6-11bbada0d7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:22.117842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e2715-7d2c-4f8c-88e6-11bbada0d7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.118114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e2715-7d2c-4f8c-88e6-11bbada0d7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.118379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e2715-7d2c-4f8c-88e6-11bbada0d7d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:22.119067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e2715-7d2c-4f8c-88e6-11bbada0d7d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:37:22.119351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e2715-7d2c-4f8c-88e6-11bbada0d7d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:22.120937 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e2715-7d2c-4f8c-88e6-11bbada0d7d3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:22.132826 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1183/2373] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:37:22.137421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20513f2-fe6d-4009-89d3-4be149f9bd56 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:22.147008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20513f2-fe6d-4009-89d3-4be149f9bd56 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:22.152579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c224f-d651-45fa-b12c-d0929791122c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:22.153169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c224f-d651-45fa-b12c-d0929791122c admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.153441 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c224f-d651-45fa-b12c-d0929791122c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.153704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c224f-d651-45fa-b12c-d0929791122c admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:22.154326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c224f-d651-45fa-b12c-d0929791122c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:22.156266 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c224f-d651-45fa-b12c-d0929791122c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:22.187537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1191/2374] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:37:22 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:37:22.191787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e2715-7d2c-4f8c-88e6-11bbada0d7d3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:22.202030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e2715-7d2c-4f8c-88e6-11bbada0d7d3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:22.258772 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f362321-9270-4623-82b9-0453f5422c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:22.259368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f362321-9270-4623-82b9-0453f5422c8c admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.259628 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f362321-9270-4623-82b9-0453f5422c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.259903 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f362321-9270-4623-82b9-0453f5422c8c admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:22.260522 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f362321-9270-4623-82b9-0453f5422c8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:22.262434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f362321-9270-4623-82b9-0453f5422c8c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:22.284657 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1184/2375] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:37:22 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:37:22.288928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c224f-d651-45fa-b12c-d0929791122c admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:22.299653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c224f-d651-45fa-b12c-d0929791122c admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:22.304216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25980406-73f1-43ef-86cc-16b145fc8d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:22.304810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980406-73f1-43ef-86cc-16b145fc8d1f admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.305084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980406-73f1-43ef-86cc-16b145fc8d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.305348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980406-73f1-43ef-86cc-16b145fc8d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:22.305986 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25980406-73f1-43ef-86cc-16b145fc8d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:22.307625 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25980406-73f1-43ef-86cc-16b145fc8d1f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:22.317793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1192/2376] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:22.321795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f362321-9270-4623-82b9-0453f5422c8c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:22.331597 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f362321-9270-4623-82b9-0453f5422c8c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:22.336620 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb65812-bd7a-497e-b174-40c2cd48b71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:22.337211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb65812-bd7a-497e-b174-40c2cd48b71e admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.337483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb65812-bd7a-497e-b174-40c2cd48b71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.337746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb65812-bd7a-497e-b174-40c2cd48b71e admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:22.338384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb65812-bd7a-497e-b174-40c2cd48b71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:22.340013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb65812-bd7a-497e-b174-40c2cd48b71e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:22.350645 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1185/2377] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:22.354763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25980406-73f1-43ef-86cc-16b145fc8d1f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:22.365039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25980406-73f1-43ef-86cc-16b145fc8d1f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:22.370134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7561bae-5d77-4f96-8e1b-bc3e9b0e12cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:22.370801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7561bae-5d77-4f96-8e1b-bc3e9b0e12cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.371075 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7561bae-5d77-4f96-8e1b-bc3e9b0e12cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.371341 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7561bae-5d77-4f96-8e1b-bc3e9b0e12cd admin admin] PATH_INFO: `/v3/projects/0881ce70083a42a5bba0fef190e2c628/users/b4d701375af2412680f8886f6f78762b/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:22.386126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7561bae-5d77-4f96-8e1b-bc3e9b0e12cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0881ce70083a42a5bba0fef190e2c628, user_id=b4d701375af2412680f8886f6f78762b, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:22.388821 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7561bae-5d77-4f96-8e1b-bc3e9b0e12cd admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:22.402539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1193/2378] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:37:22 2022] PUT /identity/v3/projects/0881ce70083a42a5bba0fef190e2c628/users/b4d701375af2412680f8886f6f78762b/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:22.414321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25884a02-e8e2-47c9-baf8-5abb924cef14 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.414642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25884a02-e8e2-47c9-baf8-5abb924cef14 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.414918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25884a02-e8e2-47c9-baf8-5abb924cef14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:22.448897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-25884a02-e8e2-47c9-baf8-5abb924cef14 None None] MFA Rules not processed for user `b4d701375af2412680f8886f6f78762b`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:22.450210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-25884a02-e8e2-47c9-baf8-5abb924cef14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:22.520616 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1186/2379] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:22.525185 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7561bae-5d77-4f96-8e1b-bc3e9b0e12cd admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:22.597246 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7561bae-5d77-4f96-8e1b-bc3e9b0e12cd admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:22.602637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e459db-f168-4045-afdb-ea3e0fcab884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:22.602991 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e459db-f168-4045-afdb-ea3e0fcab884 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.603269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e459db-f168-4045-afdb-ea3e0fcab884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.603540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e459db-f168-4045-afdb-ea3e0fcab884 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:22.604167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e459db-f168-4045-afdb-ea3e0fcab884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:22.606101 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e459db-f168-4045-afdb-ea3e0fcab884 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:22.635892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1194/2380] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:37:22 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:37:22.639871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25884a02-e8e2-47c9-baf8-5abb924cef14 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:22.650357 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25884a02-e8e2-47c9-baf8-5abb924cef14 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:22.684105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:22.714794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046641f0-2828-4e89-a428-5ff551094d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:22.715520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046641f0-2828-4e89-a428-5ff551094d2f admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.715792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046641f0-2828-4e89-a428-5ff551094d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.716062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046641f0-2828-4e89-a428-5ff551094d2f admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:22.716690 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046641f0-2828-4e89-a428-5ff551094d2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:22.718720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046641f0-2828-4e89-a428-5ff551094d2f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:22.741331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1187/2381] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:37:22 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:37:22.745610 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e459db-f168-4045-afdb-ea3e0fcab884 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:22.751173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:22.758472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e459db-f168-4045-afdb-ea3e0fcab884 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:22.764760 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425b91b-de91-4eda-ba82-1dd94781698a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:22.765430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425b91b-de91-4eda-ba82-1dd94781698a admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.765767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425b91b-de91-4eda-ba82-1dd94781698a admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.766116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425b91b-de91-4eda-ba82-1dd94781698a admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:22.766857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425b91b-de91-4eda-ba82-1dd94781698a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:22.768517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425b91b-de91-4eda-ba82-1dd94781698a admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:22.779438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1195/2382] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:22.783841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046641f0-2828-4e89-a428-5ff551094d2f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:22.796716 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046641f0-2828-4e89-a428-5ff551094d2f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:22.802213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060d6db-1756-4ba2-8630-8252275b1c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:22.803063 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060d6db-1756-4ba2-8630-8252275b1c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.803466 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060d6db-1756-4ba2-8630-8252275b1c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.803836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060d6db-1756-4ba2-8630-8252275b1c70 admin admin] PATH_INFO: `/v3/projects/ff2ca34dbd6e485d94b0e2a7526bda67/users/d39763ebb48b421cb150b74b28d93e83/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:22.820181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060d6db-1756-4ba2-8630-8252275b1c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff2ca34dbd6e485d94b0e2a7526bda67, user_id=d39763ebb48b421cb150b74b28d93e83, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:22.822891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060d6db-1756-4ba2-8630-8252275b1c70 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:22.838657 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1188/2383] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:37:22 2022] PUT /identity/v3/projects/ff2ca34dbd6e485d94b0e2a7526bda67/users/d39763ebb48b421cb150b74b28d93e83/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:22.842838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425b91b-de91-4eda-ba82-1dd94781698a admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:22.854105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425b91b-de91-4eda-ba82-1dd94781698a admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:22.913189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe369b-2f6e-4244-adb8-4e6548157896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:22.913700 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe369b-2f6e-4244-adb8-4e6548157896 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.913996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe369b-2f6e-4244-adb8-4e6548157896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.914260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe369b-2f6e-4244-adb8-4e6548157896 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:22.914978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe369b-2f6e-4244-adb8-4e6548157896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:22.916900 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe369b-2f6e-4244-adb8-4e6548157896 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:22.927472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1196/2384] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:37:22.931535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060d6db-1756-4ba2-8630-8252275b1c70 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:22.943745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060d6db-1756-4ba2-8630-8252275b1c70 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:22.948636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccab57-388e-4c67-ac96-d4031d243f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:22.949305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccab57-388e-4c67-ac96-d4031d243f94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.949577 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccab57-388e-4c67-ac96-d4031d243f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.949844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccab57-388e-4c67-ac96-d4031d243f94 admin admin] PATH_INFO: `/v3/projects/ff2ca34dbd6e485d94b0e2a7526bda67/users/d39763ebb48b421cb150b74b28d93e83/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:22.951711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ccab57-388e-4c67-ac96-d4031d243f94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff2ca34dbd6e485d94b0e2a7526bda67, user_id=d39763ebb48b421cb150b74b28d93e83, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:22.954341 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ccab57-388e-4c67-ac96-d4031d243f94 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:22.965593 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1189/2385] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:37:22 2022] PUT /identity/v3/projects/ff2ca34dbd6e485d94b0e2a7526bda67/users/d39763ebb48b421cb150b74b28d93e83/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:22.976218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102e847-d2b0-48a9-8e38-8e14e2c24e54 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:22.976590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102e847-d2b0-48a9-8e38-8e14e2c24e54 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:22.976962 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102e847-d2b0-48a9-8e38-8e14e2c24e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:23.005752 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-e102e847-d2b0-48a9-8e38-8e14e2c24e54 None None] MFA Rules not processed for user `d39763ebb48b421cb150b74b28d93e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:23.006999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e102e847-d2b0-48a9-8e38-8e14e2c24e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:23.080483 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1197/2386] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:23.088512 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccab57-388e-4c67-ac96-d4031d243f94 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:23.090272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-73ccab57-388e-4c67-ac96-d4031d243f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:23.152014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccab57-388e-4c67-ac96-d4031d243f94 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:23.156248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0f310-7eb5-48a3-b44b-38f7a6c74355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:23.157122 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0f310-7eb5-48a3-b44b-38f7a6c74355 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:23.157520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0f310-7eb5-48a3-b44b-38f7a6c74355 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:23.157915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0f310-7eb5-48a3-b44b-38f7a6c74355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:23.226472 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0f310-7eb5-48a3-b44b-38f7a6c74355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:23.227948 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0f310-7eb5-48a3-b44b-38f7a6c74355 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:23.237064 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1190/2387] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:23.744826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102e847-d2b0-48a9-8e38-8e14e2c24e54 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:23.746366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e102e847-d2b0-48a9-8e38-8e14e2c24e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:23.808475 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102e847-d2b0-48a9-8e38-8e14e2c24e54 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:23.811989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedf379-5866-49fe-bfc3-7b9fb9253a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:23.812601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedf379-5866-49fe-bfc3-7b9fb9253a74 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:23.812881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedf379-5866-49fe-bfc3-7b9fb9253a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:23.813150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedf379-5866-49fe-bfc3-7b9fb9253a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:23.815034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7fedf379-5866-49fe-bfc3-7b9fb9253a74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:23.824888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedf379-5866-49fe-bfc3-7b9fb9253a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:23.826345 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedf379-5866-49fe-bfc3-7b9fb9253a74 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:23.834511 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1198/2388] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:24.661070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0f310-7eb5-48a3-b44b-38f7a6c74355 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:24.662626 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-95c0f310-7eb5-48a3-b44b-38f7a6c74355 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:24.721339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:24.756980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0f310-7eb5-48a3-b44b-38f7a6c74355 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:24.765647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ed4a6-8352-429d-beae-a3d6106c39e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:24.765647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ed4a6-8352-429d-beae-a3d6106c39e0 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:24.765647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ed4a6-8352-429d-beae-a3d6106c39e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:24.765647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ed4a6-8352-429d-beae-a3d6106c39e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:24.767029 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-463ed4a6-8352-429d-beae-a3d6106c39e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:24.834716 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ed4a6-8352-429d-beae-a3d6106c39e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:24.836249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ed4a6-8352-429d-beae-a3d6106c39e0 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:24.859258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1191/2389] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:27.917159 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedf379-5866-49fe-bfc3-7b9fb9253a74 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:27.927114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedf379-5866-49fe-bfc3-7b9fb9253a74 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:27.931492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920b0ef-7c3d-4caf-86df-bcb603191634 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:27.932093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920b0ef-7c3d-4caf-86df-bcb603191634 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:27.932369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920b0ef-7c3d-4caf-86df-bcb603191634 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:27.932642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920b0ef-7c3d-4caf-86df-bcb603191634 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:27.942160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920b0ef-7c3d-4caf-86df-bcb603191634 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:27.943623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920b0ef-7c3d-4caf-86df-bcb603191634 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:27.952997 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1199/2390] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:34.841484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=37 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:37.949188 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:53.069317 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ed4a6-8352-429d-beae-a3d6106c39e0 service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:53.137611 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ed4a6-8352-429d-beae-a3d6106c39e0 service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:53.142070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1d75a-170b-4bdb-ad8a-56308ac51b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:53.142747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1d75a-170b-4bdb-ad8a-56308ac51b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:53.143217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1d75a-170b-4bdb-ad8a-56308ac51b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:53.143496 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1d75a-170b-4bdb-ad8a-56308ac51b83 admin admin] PATH_INFO: `/v3/users/b4d701375af2412680f8886f6f78762b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:53.144639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1d75a-170b-4bdb-ad8a-56308ac51b83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4d701375af2412680f8886f6f78762b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:53.146573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1d75a-170b-4bdb-ad8a-56308ac51b83 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:53.209778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-29d1d75a-170b-4bdb-ad8a-56308ac51b83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4d701375af2412680f8886f6f78762b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.209778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-29d1d75a-170b-4bdb-ad8a-56308ac51b83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4d701375af2412680f8886f6f78762b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.209778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-29d1d75a-170b-4bdb-ad8a-56308ac51b83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4d701375af2412680f8886f6f78762b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.209778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-29d1d75a-170b-4bdb-ad8a-56308ac51b83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4d701375af2412680f8886f6f78762b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.215057 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-29d1d75a-170b-4bdb-ad8a-56308ac51b83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4d701375af2412680f8886f6f78762b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.229672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1192/2391] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:37:53 2022] DELETE /identity/v3/users/b4d701375af2412680f8886f6f78762b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:53.234614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0793703-84e8-44cf-96a6-5a3d75a7d9aa None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:53.234614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0793703-84e8-44cf-96a6-5a3d75a7d9aa None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:53.235171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0793703-84e8-44cf-96a6-5a3d75a7d9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:53.257151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d0793703-84e8-44cf-96a6-5a3d75a7d9aa None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:53.257151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d0793703-84e8-44cf-96a6-5a3d75a7d9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:53.314753 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1200/2392] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:53.324558 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1d75a-170b-4bdb-ad8a-56308ac51b83 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:53.326054 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-29d1d75a-170b-4bdb-ad8a-56308ac51b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:53.389734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1d75a-170b-4bdb-ad8a-56308ac51b83 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:53.393479 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f86175-cde3-404a-8502-d432ba53f3d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:53.394090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f86175-cde3-404a-8502-d432ba53f3d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:53.394368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f86175-cde3-404a-8502-d432ba53f3d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:53.394727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f86175-cde3-404a-8502-d432ba53f3d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:53.403887 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f86175-cde3-404a-8502-d432ba53f3d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:53.405319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f86175-cde3-404a-8502-d432ba53f3d5 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:53.415107 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1193/2393] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:53.716798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0793703-84e8-44cf-96a6-5a3d75a7d9aa None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:53.718364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d0793703-84e8-44cf-96a6-5a3d75a7d9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:53.730472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0793703-84e8-44cf-96a6-5a3d75a7d9aa None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:53.735127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:53.735768 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:53.736041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:53.736310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] PATH_INFO: `/v3/projects/0881ce70083a42a5bba0fef190e2c628` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:53.737422 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0881ce70083a42a5bba0fef190e2c628)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:53.739343 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:53.785884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0881ce70083a42a5bba0fef190e2c628', 'request_id': 'req-750b4163-d4dd-42e1-931d-664237e62b98'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.792455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0881ce70083a42a5bba0fef190e2c628', 'request_id': 'req-750b4163-d4dd-42e1-931d-664237e62b98'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.801833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0881ce70083a42a5bba0fef190e2c628', 'request_id': 'req-750b4163-d4dd-42e1-931d-664237e62b98'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.812411 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] The token cache is being invalidate because project 0881ce70083a42a5bba0fef190e2c628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:37:53.812693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.813321 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.814892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1201/2394] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:37:53 2022] DELETE /identity/v3/projects/0881ce70083a42a5bba0fef190e2c628 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:53.819016 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f86175-cde3-404a-8502-d432ba53f3d5 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:53.828787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f86175-cde3-404a-8502-d432ba53f3d5 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:53.847867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:53.889040 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059c742-7a9f-4322-b4cc-ba456638f77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:53.889638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059c742-7a9f-4322-b4cc-ba456638f77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:53.889916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059c742-7a9f-4322-b4cc-ba456638f77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:53.890189 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059c742-7a9f-4322-b4cc-ba456638f77f admin admin] PATH_INFO: `/v3/users/d39763ebb48b421cb150b74b28d93e83` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:53.891383 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059c742-7a9f-4322-b4cc-ba456638f77f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39763ebb48b421cb150b74b28d93e83)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:53.893315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059c742-7a9f-4322-b4cc-ba456638f77f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:53.946754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4059c742-7a9f-4322-b4cc-ba456638f77f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39763ebb48b421cb150b74b28d93e83'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.947415 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4059c742-7a9f-4322-b4cc-ba456638f77f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39763ebb48b421cb150b74b28d93e83'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.948026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4059c742-7a9f-4322-b4cc-ba456638f77f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39763ebb48b421cb150b74b28d93e83'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.955018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4059c742-7a9f-4322-b4cc-ba456638f77f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39763ebb48b421cb150b74b28d93e83'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.974451 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4059c742-7a9f-4322-b4cc-ba456638f77f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39763ebb48b421cb150b74b28d93e83'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:53.993803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1194/2395] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:37:53 2022] DELETE /identity/v3/users/d39763ebb48b421cb150b74b28d93e83 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:54.280638 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:54.282799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:54.299430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b4163-d4dd-42e1-931d-664237e62b98 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:54.356658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a312c46-cbdf-4370-9503-15e1c7505e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:54.357290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a312c46-cbdf-4370-9503-15e1c7505e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:54.357561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a312c46-cbdf-4370-9503-15e1c7505e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:54.357936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a312c46-cbdf-4370-9503-15e1c7505e4c admin admin] PATH_INFO: `/v3/projects/ff2ca34dbd6e485d94b0e2a7526bda67` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:54.359121 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a312c46-cbdf-4370-9503-15e1c7505e4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff2ca34dbd6e485d94b0e2a7526bda67)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:54.361014 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a312c46-cbdf-4370-9503-15e1c7505e4c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:54.393474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:54.410826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4a312c46-cbdf-4370-9503-15e1c7505e4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff2ca34dbd6e485d94b0e2a7526bda67', 'request_id': 'req-4a312c46-cbdf-4370-9503-15e1c7505e4c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:54.417374 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4a312c46-cbdf-4370-9503-15e1c7505e4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff2ca34dbd6e485d94b0e2a7526bda67', 'request_id': 'req-4a312c46-cbdf-4370-9503-15e1c7505e4c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:54.425827 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4a312c46-cbdf-4370-9503-15e1c7505e4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff2ca34dbd6e485d94b0e2a7526bda67', 'request_id': 'req-4a312c46-cbdf-4370-9503-15e1c7505e4c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:54.436989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4a312c46-cbdf-4370-9503-15e1c7505e4c admin admin] The token cache is being invalidate because project ff2ca34dbd6e485d94b0e2a7526bda67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:37:54.437275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4a312c46-cbdf-4370-9503-15e1c7505e4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:54.437895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4a312c46-cbdf-4370-9503-15e1c7505e4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:37:54.439441 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1202/2396] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:37:54 2022] DELETE /identity/v3/projects/ff2ca34dbd6e485d94b0e2a7526bda67 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:54.444536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fc0fa-c69d-4a97-8c42-c4a2aa4409ef None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:54.445069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fc0fa-c69d-4a97-8c42-c4a2aa4409ef None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:54.445069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fc0fa-c69d-4a97-8c42-c4a2aa4409ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:54.466150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-409fc0fa-c69d-4a97-8c42-c4a2aa4409ef None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:54.467379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-409fc0fa-c69d-4a97-8c42-c4a2aa4409ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:54.528381 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1195/2397] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:54.537581 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea915b-24aa-44d6-bbc3-d23d086fc84c None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:54.537856 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea915b-24aa-44d6-bbc3-d23d086fc84c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:54.538156 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea915b-24aa-44d6-bbc3-d23d086fc84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:54.559103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-99ea915b-24aa-44d6-bbc3-d23d086fc84c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:54.560275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-99ea915b-24aa-44d6-bbc3-d23d086fc84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:54.568752 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1203/2398] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:37:54.575304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fc0fa-c69d-4a97-8c42-c4a2aa4409ef None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:54.587551 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fc0fa-c69d-4a97-8c42-c4a2aa4409ef None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:54.592943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad3ad3-795d-4d33-a25a-198bd081a198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:54.593544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad3ad3-795d-4d33-a25a-198bd081a198 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:54.593822 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad3ad3-795d-4d33-a25a-198bd081a198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:54.594080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad3ad3-795d-4d33-a25a-198bd081a198 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:54.594746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad3ad3-795d-4d33-a25a-198bd081a198 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:37:54.595041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad3ad3-795d-4d33-a25a-198bd081a198 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:54.596637 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad3ad3-795d-4d33-a25a-198bd081a198 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:54.609585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1196/2399] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:37:54.616833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea915b-24aa-44d6-bbc3-d23d086fc84c None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:54.626381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea915b-24aa-44d6-bbc3-d23d086fc84c None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:54.635761 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fb8d9-f653-498e-aacd-082f234cc0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:54.636369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fb8d9-f653-498e-aacd-082f234cc0e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:54.636643 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fb8d9-f653-498e-aacd-082f234cc0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:54.636910 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fb8d9-f653-498e-aacd-082f234cc0e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:54.637533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449fb8d9-f653-498e-aacd-082f234cc0e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:54.639545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449fb8d9-f653-498e-aacd-082f234cc0e1 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:54.670388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1204/2400] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:37:54 2022] POST /identity/v3/projects => generated 409 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:37:54.675344 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad3ad3-795d-4d33-a25a-198bd081a198 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:54.688855 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad3ad3-795d-4d33-a25a-198bd081a198 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:54.717248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:54.750938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecec24-64e0-4efb-b355-b675bde6f6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:54.751544 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecec24-64e0-4efb-b355-b675bde6f6bd admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:54.751793 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecec24-64e0-4efb-b355-b675bde6f6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:54.752066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecec24-64e0-4efb-b355-b675bde6f6bd admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:54.752683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecec24-64e0-4efb-b355-b675bde6f6bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:54.754589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecec24-64e0-4efb-b355-b675bde6f6bd admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:54.780541 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1197/2401] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:37:54 2022] POST /identity/v3/users => generated 305 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:37:54.782197 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fb8d9-f653-498e-aacd-082f234cc0e1 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:54.792958 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fb8d9-f653-498e-aacd-082f234cc0e1 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:54.797784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279711e-99c5-4cc7-b7f6-eeeb00a577b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:54.798373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279711e-99c5-4cc7-b7f6-eeeb00a577b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:54.798701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279711e-99c5-4cc7-b7f6-eeeb00a577b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:54.799017 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279711e-99c5-4cc7-b7f6-eeeb00a577b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:54.799663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279711e-99c5-4cc7-b7f6-eeeb00a577b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:54.801315 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279711e-99c5-4cc7-b7f6-eeeb00a577b4 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:54.811659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1205/2402] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:54.815697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecec24-64e0-4efb-b355-b675bde6f6bd admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:54.825859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecec24-64e0-4efb-b355-b675bde6f6bd admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:54.830619 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ae00c-fe59-4f4d-a5ea-1dc1d787eaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:54.831223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ae00c-fe59-4f4d-a5ea-1dc1d787eaf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:54.831494 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ae00c-fe59-4f4d-a5ea-1dc1d787eaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:54.831759 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ae00c-fe59-4f4d-a5ea-1dc1d787eaf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:54.832653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ae00c-fe59-4f4d-a5ea-1dc1d787eaf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:54.834263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ae00c-fe59-4f4d-a5ea-1dc1d787eaf1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:54.844238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1198/2403] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:37:54.848517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279711e-99c5-4cc7-b7f6-eeeb00a577b4 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:54.858522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279711e-99c5-4cc7-b7f6-eeeb00a577b4 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:54.868836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e4346-c89f-4239-a65b-b348fca878b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:54.869591 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e4346-c89f-4239-a65b-b348fca878b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:54.869869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e4346-c89f-4239-a65b-b348fca878b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:54.870137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e4346-c89f-4239-a65b-b348fca878b6 admin admin] PATH_INFO: `/v3/projects/ef72db23aabf4389af2f1f153358a29e/users/3148b0b51bec485680883847d7fd0f19/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:54.886190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e4346-c89f-4239-a65b-b348fca878b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef72db23aabf4389af2f1f153358a29e, user_id=3148b0b51bec485680883847d7fd0f19, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:54.888917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e4346-c89f-4239-a65b-b348fca878b6 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:54.903601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1206/2404] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:37:54 2022] PUT /identity/v3/projects/ef72db23aabf4389af2f1f153358a29e/users/3148b0b51bec485680883847d7fd0f19/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:37:54.914316 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a01ca-09c7-4685-81a8-785012bcd66d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:54.915004 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a01ca-09c7-4685-81a8-785012bcd66d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:54.915275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a01ca-09c7-4685-81a8-785012bcd66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:54.943049 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-6d6a01ca-09c7-4685-81a8-785012bcd66d None None] MFA Rules not processed for user `3148b0b51bec485680883847d7fd0f19`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:54.944275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6d6a01ca-09c7-4685-81a8-785012bcd66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:55.022786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1199/2405] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:54 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:55.029632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e4346-c89f-4239-a65b-b348fca878b6 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:55.106989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e4346-c89f-4239-a65b-b348fca878b6 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:55.112258 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c18e4-9c18-4d2b-8e27-5a72025fa88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:55.112866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c18e4-9c18-4d2b-8e27-5a72025fa88f admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:55.113140 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c18e4-9c18-4d2b-8e27-5a72025fa88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:55.113407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c18e4-9c18-4d2b-8e27-5a72025fa88f admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:55.114039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c18e4-9c18-4d2b-8e27-5a72025fa88f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:55.115982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c18e4-9c18-4d2b-8e27-5a72025fa88f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:55.145348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1207/2406] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:37:55 2022] POST /identity/v3/projects => generated 409 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:37:55.149622 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a01ca-09c7-4685-81a8-785012bcd66d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:55.160853 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a01ca-09c7-4685-81a8-785012bcd66d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:55.220030 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b700f85-b9da-4090-86a1-83a2ae14c8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:55.220638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b700f85-b9da-4090-86a1-83a2ae14c8fe admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:55.220914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b700f85-b9da-4090-86a1-83a2ae14c8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:55.221176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b700f85-b9da-4090-86a1-83a2ae14c8fe admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:55.221830 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b700f85-b9da-4090-86a1-83a2ae14c8fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:55.223876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b700f85-b9da-4090-86a1-83a2ae14c8fe admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:55.247037 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1200/2407] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:37:55 2022] POST /identity/v3/users => generated 311 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:37:55.251291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c18e4-9c18-4d2b-8e27-5a72025fa88f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:55.261732 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c18e4-9c18-4d2b-8e27-5a72025fa88f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:55.266472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc56e9c-2a9c-4836-be0b-73406fdea8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:55.267100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc56e9c-2a9c-4836-be0b-73406fdea8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:55.267380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc56e9c-2a9c-4836-be0b-73406fdea8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:55.267648 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc56e9c-2a9c-4836-be0b-73406fdea8b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:55.268333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc56e9c-2a9c-4836-be0b-73406fdea8b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:55.269980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc56e9c-2a9c-4836-be0b-73406fdea8b0 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:55.281619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1208/2408] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:55.286074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b700f85-b9da-4090-86a1-83a2ae14c8fe admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:55.297158 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b700f85-b9da-4090-86a1-83a2ae14c8fe admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:55.302798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd813919-5cd9-4e7a-81fd-cb41554041be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:55.303454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd813919-5cd9-4e7a-81fd-cb41554041be admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:55.303726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd813919-5cd9-4e7a-81fd-cb41554041be admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:55.303989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd813919-5cd9-4e7a-81fd-cb41554041be admin admin] PATH_INFO: `/v3/projects/2fbc1d01151747a7aec6cdb3558ccf42/users/00dc440a28e74138aab1e085efe4bdb3/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:55.319754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd813919-5cd9-4e7a-81fd-cb41554041be admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fbc1d01151747a7aec6cdb3558ccf42, user_id=00dc440a28e74138aab1e085efe4bdb3, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:55.322428 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd813919-5cd9-4e7a-81fd-cb41554041be admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:55.338107 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1201/2409] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:37:55 2022] PUT /identity/v3/projects/2fbc1d01151747a7aec6cdb3558ccf42/users/00dc440a28e74138aab1e085efe4bdb3/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:55.342230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc56e9c-2a9c-4836-be0b-73406fdea8b0 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:55.352740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc56e9c-2a9c-4836-be0b-73406fdea8b0 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:55.372796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:55.413556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2e34e-98b3-4494-8213-77629a655308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:55.414180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2e34e-98b3-4494-8213-77629a655308 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:55.414456 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2e34e-98b3-4494-8213-77629a655308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:55.415124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2e34e-98b3-4494-8213-77629a655308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:55.415770 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f2e34e-98b3-4494-8213-77629a655308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:55.417366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f2e34e-98b3-4494-8213-77629a655308 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:55.426925 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1209/2410] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:37:55.431467 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd813919-5cd9-4e7a-81fd-cb41554041be admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:55.437305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:37:55.444060 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd813919-5cd9-4e7a-81fd-cb41554041be admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:55.449683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3668d-76be-4732-b6a7-7fed5f1df80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:55.450323 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3668d-76be-4732-b6a7-7fed5f1df80e admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:55.450640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3668d-76be-4732-b6a7-7fed5f1df80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:55.450911 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3668d-76be-4732-b6a7-7fed5f1df80e admin admin] PATH_INFO: `/v3/projects/2fbc1d01151747a7aec6cdb3558ccf42/users/00dc440a28e74138aab1e085efe4bdb3/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:55.452882 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3668d-76be-4732-b6a7-7fed5f1df80e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fbc1d01151747a7aec6cdb3558ccf42, user_id=00dc440a28e74138aab1e085efe4bdb3, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:55.455685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3668d-76be-4732-b6a7-7fed5f1df80e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:55.465958 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1202/2411] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:37:55 2022] PUT /identity/v3/projects/2fbc1d01151747a7aec6cdb3558ccf42/users/00dc440a28e74138aab1e085efe4bdb3/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:37:55.476390 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6bdea-5b3a-408e-b1d5-6854f0003deb None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:55.476667 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6bdea-5b3a-408e-b1d5-6854f0003deb None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:55.476937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6bdea-5b3a-408e-b1d5-6854f0003deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:55.507102 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-b8a6bdea-5b3a-408e-b1d5-6854f0003deb None None] MFA Rules not processed for user `00dc440a28e74138aab1e085efe4bdb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:37:55.508322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b8a6bdea-5b3a-408e-b1d5-6854f0003deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:55.579779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1210/2412] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:37:55 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:37:55.591283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3668d-76be-4732-b6a7-7fed5f1df80e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:55.593089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-fad3668d-76be-4732-b6a7-7fed5f1df80e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:55.654401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3668d-76be-4732-b6a7-7fed5f1df80e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:55.658418 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0dedf-0a2d-4e7f-9fa8-f6063755afe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:55.659111 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0dedf-0a2d-4e7f-9fa8-f6063755afe5 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:55.659396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0dedf-0a2d-4e7f-9fa8-f6063755afe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:55.659668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0dedf-0a2d-4e7f-9fa8-f6063755afe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:55.670304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0dedf-0a2d-4e7f-9fa8-f6063755afe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:55.671800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0dedf-0a2d-4e7f-9fa8-f6063755afe5 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:55.681586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1203/2413] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:55 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:55.839975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6bdea-5b3a-408e-b1d5-6854f0003deb None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:55.850029 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6bdea-5b3a-408e-b1d5-6854f0003deb None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:55.854325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6db46f-9422-4b9e-ab31-6b6ea27e518d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:55.854972 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6db46f-9422-4b9e-ab31-6b6ea27e518d service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:55.855243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6db46f-9422-4b9e-ab31-6b6ea27e518d service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:55.855566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6db46f-9422-4b9e-ab31-6b6ea27e518d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:55.916582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6db46f-9422-4b9e-ab31-6b6ea27e518d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:55.918088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6db46f-9422-4b9e-ab31-6b6ea27e518d service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:55.927669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1211/2414] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:55 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:37:57.352852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0dedf-0a2d-4e7f-9fa8-f6063755afe5 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:37:57.357641 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a9f0dedf-0a2d-4e7f-9fa8-f6063755afe5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:57.513287 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0dedf-0a2d-4e7f-9fa8-f6063755afe5 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:37:57.520858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611553f-cc8a-4f91-8bd6-6846d8989599 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:37:57.521504 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611553f-cc8a-4f91-8bd6-6846d8989599 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:37:57.521789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611553f-cc8a-4f91-8bd6-6846d8989599 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:37:57.522056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611553f-cc8a-4f91-8bd6-6846d8989599 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:37:57.524114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b611553f-cc8a-4f91-8bd6-6846d8989599 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:37:57.539950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611553f-cc8a-4f91-8bd6-6846d8989599 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:37:57.541722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611553f-cc8a-4f91-8bd6-6846d8989599 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:37:57.555172 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1204/2415] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:37:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:03.476673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6db46f-9422-4b9e-ab31-6b6ea27e518d service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:03.521287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:03.540389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6db46f-9422-4b9e-ab31-6b6ea27e518d service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:03.545528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef0d03-7815-4a36-abe4-7634e3e69680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:03.546158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef0d03-7815-4a36-abe4-7634e3e69680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:03.546435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef0d03-7815-4a36-abe4-7634e3e69680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:03.546733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef0d03-7815-4a36-abe4-7634e3e69680 admin admin] PATH_INFO: `/v3/users/3148b0b51bec485680883847d7fd0f19` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:03.547918 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef0d03-7815-4a36-abe4-7634e3e69680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3148b0b51bec485680883847d7fd0f19)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:03.549778 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef0d03-7815-4a36-abe4-7634e3e69680 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:03.598582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d9ef0d03-7815-4a36-abe4-7634e3e69680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3148b0b51bec485680883847d7fd0f19'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:03.599299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d9ef0d03-7815-4a36-abe4-7634e3e69680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3148b0b51bec485680883847d7fd0f19'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:03.600022 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d9ef0d03-7815-4a36-abe4-7634e3e69680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3148b0b51bec485680883847d7fd0f19'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:03.607086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d9ef0d03-7815-4a36-abe4-7634e3e69680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3148b0b51bec485680883847d7fd0f19'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:03.621132 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d9ef0d03-7815-4a36-abe4-7634e3e69680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3148b0b51bec485680883847d7fd0f19'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:03.635097 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1212/2416] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:03 2022] DELETE /identity/v3/users/3148b0b51bec485680883847d7fd0f19 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:03.640673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91316c10-de57-40ec-bbce-cfda5f58885e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:03.641177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91316c10-de57-40ec-bbce-cfda5f58885e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:03.641285 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91316c10-de57-40ec-bbce-cfda5f58885e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:03.663525 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-91316c10-de57-40ec-bbce-cfda5f58885e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:03.665161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-91316c10-de57-40ec-bbce-cfda5f58885e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:03.725679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1205/2417] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:03.735880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef0d03-7815-4a36-abe4-7634e3e69680 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:03.737829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d9ef0d03-7815-4a36-abe4-7634e3e69680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:03.809051 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef0d03-7815-4a36-abe4-7634e3e69680 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:03.814156 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753a544-4ed2-4603-9564-d5c5525b7bb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:03.815035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753a544-4ed2-4603-9564-d5c5525b7bb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:03.815310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753a544-4ed2-4603-9564-d5c5525b7bb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:03.815659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753a544-4ed2-4603-9564-d5c5525b7bb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:03.831728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753a544-4ed2-4603-9564-d5c5525b7bb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:03.833800 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753a544-4ed2-4603-9564-d5c5525b7bb1 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:03.843830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1213/2418] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:04.121633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91316c10-de57-40ec-bbce-cfda5f58885e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:04.123200 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-91316c10-de57-40ec-bbce-cfda5f58885e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:04.136453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91316c10-de57-40ec-bbce-cfda5f58885e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:04.141765 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:04.142427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:04.142746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:04.143014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] PATH_INFO: `/v3/projects/ef72db23aabf4389af2f1f153358a29e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:04.144281 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef72db23aabf4389af2f1f153358a29e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:04.146222 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:04.158442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:04.196083 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef72db23aabf4389af2f1f153358a29e', 'request_id': 'req-caa49f51-fe85-4664-8e69-d680a30bd689'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.203922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef72db23aabf4389af2f1f153358a29e', 'request_id': 'req-caa49f51-fe85-4664-8e69-d680a30bd689'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.213124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef72db23aabf4389af2f1f153358a29e', 'request_id': 'req-caa49f51-fe85-4664-8e69-d680a30bd689'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.230458 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] The token cache is being invalidate because project ef72db23aabf4389af2f1f153358a29e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:04.230783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.231416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.232986 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1206/2419] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:04 2022] DELETE /identity/v3/projects/ef72db23aabf4389af2f1f153358a29e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:04.237076 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753a544-4ed2-4603-9564-d5c5525b7bb1 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:04.247314 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753a544-4ed2-4603-9564-d5c5525b7bb1 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:04.305167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce145d-600b-4493-97b2-4b719f803015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:04.305812 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce145d-600b-4493-97b2-4b719f803015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:04.306091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce145d-600b-4493-97b2-4b719f803015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:04.306359 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce145d-600b-4493-97b2-4b719f803015 admin admin] PATH_INFO: `/v3/users/00dc440a28e74138aab1e085efe4bdb3` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:04.307634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce145d-600b-4493-97b2-4b719f803015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00dc440a28e74138aab1e085efe4bdb3)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:04.309563 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce145d-600b-4493-97b2-4b719f803015 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:04.315312 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:04.358158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-67ce145d-600b-4493-97b2-4b719f803015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00dc440a28e74138aab1e085efe4bdb3'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.358821 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-67ce145d-600b-4493-97b2-4b719f803015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00dc440a28e74138aab1e085efe4bdb3'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.359414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-67ce145d-600b-4493-97b2-4b719f803015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00dc440a28e74138aab1e085efe4bdb3'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.366191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-67ce145d-600b-4493-97b2-4b719f803015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00dc440a28e74138aab1e085efe4bdb3'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.378765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-67ce145d-600b-4493-97b2-4b719f803015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00dc440a28e74138aab1e085efe4bdb3'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.392160 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1214/2420] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:04 2022] DELETE /identity/v3/users/00dc440a28e74138aab1e085efe4bdb3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:04.684959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:04.686420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:04.706363 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa49f51-fe85-4664-8e69-d680a30bd689 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:04.764240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59625953-7eb6-462d-bd37-26d2562f5673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:04.764875 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59625953-7eb6-462d-bd37-26d2562f5673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:04.765151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59625953-7eb6-462d-bd37-26d2562f5673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:04.765417 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59625953-7eb6-462d-bd37-26d2562f5673 admin admin] PATH_INFO: `/v3/projects/2fbc1d01151747a7aec6cdb3558ccf42` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:04.766564 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59625953-7eb6-462d-bd37-26d2562f5673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fbc1d01151747a7aec6cdb3558ccf42)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:04.768516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59625953-7eb6-462d-bd37-26d2562f5673 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:04.820559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-59625953-7eb6-462d-bd37-26d2562f5673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fbc1d01151747a7aec6cdb3558ccf42', 'request_id': 'req-59625953-7eb6-462d-bd37-26d2562f5673'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.826769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-59625953-7eb6-462d-bd37-26d2562f5673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fbc1d01151747a7aec6cdb3558ccf42', 'request_id': 'req-59625953-7eb6-462d-bd37-26d2562f5673'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.832325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-59625953-7eb6-462d-bd37-26d2562f5673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fbc1d01151747a7aec6cdb3558ccf42', 'request_id': 'req-59625953-7eb6-462d-bd37-26d2562f5673'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.845787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-59625953-7eb6-462d-bd37-26d2562f5673 admin admin] The token cache is being invalidate because project 2fbc1d01151747a7aec6cdb3558ccf42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:04.846080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-59625953-7eb6-462d-bd37-26d2562f5673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.846830 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-59625953-7eb6-462d-bd37-26d2562f5673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:04.848389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1207/2421] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:04 2022] DELETE /identity/v3/projects/2fbc1d01151747a7aec6cdb3558ccf42 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:04.853779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ea25e-6944-4867-a256-f298902dcb74 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:04.854304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ea25e-6944-4867-a256-f298902dcb74 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:04.854304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ea25e-6944-4867-a256-f298902dcb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:04.875092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-7f6ea25e-6944-4867-a256-f298902dcb74 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:04.876301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7f6ea25e-6944-4867-a256-f298902dcb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:04.935106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1215/2422] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:05.001683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90830e82-226b-46d5-8b12-b934790b36bb None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.001984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90830e82-226b-46d5-8b12-b934790b36bb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.002224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90830e82-226b-46d5-8b12-b934790b36bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.031011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-90830e82-226b-46d5-8b12-b934790b36bb None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:05.032812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-90830e82-226b-46d5-8b12-b934790b36bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:05.044637 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1208/2423] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:05.050364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ea25e-6944-4867-a256-f298902dcb74 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:05.060326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ea25e-6944-4867-a256-f298902dcb74 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:05.066641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72904e6d-2016-4b5a-96f3-3a0f39b47bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:05.067416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72904e6d-2016-4b5a-96f3-3a0f39b47bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.067757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72904e6d-2016-4b5a-96f3-3a0f39b47bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.068081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72904e6d-2016-4b5a-96f3-3a0f39b47bf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.069010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72904e6d-2016-4b5a-96f3-3a0f39b47bf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:38:05.069406 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72904e6d-2016-4b5a-96f3-3a0f39b47bf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:05.071375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72904e6d-2016-4b5a-96f3-3a0f39b47bf4 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:05.083907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1216/2424] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:38:05.088506 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90830e82-226b-46d5-8b12-b934790b36bb None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:05.098221 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90830e82-226b-46d5-8b12-b934790b36bb None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:05.104314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5435c7d-775d-45b8-81a4-4ad947020295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:05.105000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5435c7d-775d-45b8-81a4-4ad947020295 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.105296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5435c7d-775d-45b8-81a4-4ad947020295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.105566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5435c7d-775d-45b8-81a4-4ad947020295 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.106187 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5435c7d-775d-45b8-81a4-4ad947020295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:05.108192 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5435c7d-775d-45b8-81a4-4ad947020295 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:05.142388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1209/2425] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:05 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:05.148410 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72904e6d-2016-4b5a-96f3-3a0f39b47bf4 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:05.158906 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72904e6d-2016-4b5a-96f3-3a0f39b47bf4 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:05.221654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6db2a6-7d50-44a1-af5e-8a99f728567c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:05.222274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6db2a6-7d50-44a1-af5e-8a99f728567c admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.222579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6db2a6-7d50-44a1-af5e-8a99f728567c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.222821 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6db2a6-7d50-44a1-af5e-8a99f728567c admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.223463 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6db2a6-7d50-44a1-af5e-8a99f728567c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:05.225607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6db2a6-7d50-44a1-af5e-8a99f728567c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:05.247361 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1217/2426] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:05 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:05.253016 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5435c7d-775d-45b8-81a4-4ad947020295 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:05.258775 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:05.266013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5435c7d-775d-45b8-81a4-4ad947020295 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:05.274189 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b358a-d449-49de-a411-591fbcd06b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:05.274830 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b358a-d449-49de-a411-591fbcd06b00 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.275108 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b358a-d449-49de-a411-591fbcd06b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.275379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b358a-d449-49de-a411-591fbcd06b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.276022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b358a-d449-49de-a411-591fbcd06b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:05.277760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b358a-d449-49de-a411-591fbcd06b00 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:05.288172 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1210/2427] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:05.292577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6db2a6-7d50-44a1-af5e-8a99f728567c admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:05.298037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:05.304169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6db2a6-7d50-44a1-af5e-8a99f728567c admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:05.311663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e30b9-5eef-4629-a69a-0ad1417b1a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:05.312599 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e30b9-5eef-4629-a69a-0ad1417b1a68 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.313109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e30b9-5eef-4629-a69a-0ad1417b1a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.313535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e30b9-5eef-4629-a69a-0ad1417b1a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.314557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e30b9-5eef-4629-a69a-0ad1417b1a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:05.317408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e30b9-5eef-4629-a69a-0ad1417b1a68 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:05.334563 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1218/2428] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:05.338892 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b358a-d449-49de-a411-591fbcd06b00 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:05.349674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b358a-d449-49de-a411-591fbcd06b00 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:05.355614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a396f9-3913-47a6-92d6-29722a077c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:05.356265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a396f9-3913-47a6-92d6-29722a077c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.356542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a396f9-3913-47a6-92d6-29722a077c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.356840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a396f9-3913-47a6-92d6-29722a077c10 admin admin] PATH_INFO: `/v3/projects/dcb3a783282a45fcaaed58bf79841a3c/users/3d7f9d9480ca4832b09348688d1ae0b4/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.373760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a396f9-3913-47a6-92d6-29722a077c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcb3a783282a45fcaaed58bf79841a3c, user_id=3d7f9d9480ca4832b09348688d1ae0b4, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:05.376511 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a396f9-3913-47a6-92d6-29722a077c10 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:05.392809 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1211/2429] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:05 2022] PUT /identity/v3/projects/dcb3a783282a45fcaaed58bf79841a3c/users/3d7f9d9480ca4832b09348688d1ae0b4/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:05.403354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b21dc-cc47-4c3d-999d-5c82b00b27c6 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.403637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b21dc-cc47-4c3d-999d-5c82b00b27c6 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.403904 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b21dc-cc47-4c3d-999d-5c82b00b27c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.439894 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-5d0b21dc-cc47-4c3d-999d-5c82b00b27c6 None None] MFA Rules not processed for user `3d7f9d9480ca4832b09348688d1ae0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:05.441103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5d0b21dc-cc47-4c3d-999d-5c82b00b27c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:05.505543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1219/2430] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:05.510896 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a396f9-3913-47a6-92d6-29722a077c10 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:05.578117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a396f9-3913-47a6-92d6-29722a077c10 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:05.583722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58834a29-c86f-4e9f-8b66-5778e410ade6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:05.584325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58834a29-c86f-4e9f-8b66-5778e410ade6 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.584665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58834a29-c86f-4e9f-8b66-5778e410ade6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.584846 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58834a29-c86f-4e9f-8b66-5778e410ade6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.585488 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58834a29-c86f-4e9f-8b66-5778e410ade6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:05.587438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58834a29-c86f-4e9f-8b66-5778e410ade6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:05.615498 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1212/2431] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:05 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:05.619804 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b21dc-cc47-4c3d-999d-5c82b00b27c6 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:05.631170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b21dc-cc47-4c3d-999d-5c82b00b27c6 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:05.689166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19991270-6c5a-4268-aa00-ca532741efa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:05.689774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19991270-6c5a-4268-aa00-ca532741efa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.690048 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19991270-6c5a-4268-aa00-ca532741efa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.690310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19991270-6c5a-4268-aa00-ca532741efa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.690961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19991270-6c5a-4268-aa00-ca532741efa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:05.692870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19991270-6c5a-4268-aa00-ca532741efa0 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:05.715820 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1220/2432] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:05 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:05.720039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58834a29-c86f-4e9f-8b66-5778e410ade6 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:05.730977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58834a29-c86f-4e9f-8b66-5778e410ade6 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:05.736526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c51a55-c71e-495f-9541-0ab47c5d38dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:05.737135 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c51a55-c71e-495f-9541-0ab47c5d38dd admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.737410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c51a55-c71e-495f-9541-0ab47c5d38dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.737675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c51a55-c71e-495f-9541-0ab47c5d38dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.738311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c51a55-c71e-495f-9541-0ab47c5d38dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:05.739971 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c51a55-c71e-495f-9541-0ab47c5d38dd admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:05.753346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1213/2433] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:05.757725 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19991270-6c5a-4268-aa00-ca532741efa0 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:05.771060 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19991270-6c5a-4268-aa00-ca532741efa0 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:05.778292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1a8b5-60c3-427d-a084-acb54f55a564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:05.778292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1a8b5-60c3-427d-a084-acb54f55a564 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.778292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1a8b5-60c3-427d-a084-acb54f55a564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.778292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1a8b5-60c3-427d-a084-acb54f55a564 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.779814 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1a8b5-60c3-427d-a084-acb54f55a564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:05.781316 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1a8b5-60c3-427d-a084-acb54f55a564 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:05.791712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1221/2434] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:05.796452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c51a55-c71e-495f-9541-0ab47c5d38dd admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:05.806786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c51a55-c71e-495f-9541-0ab47c5d38dd admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:05.816722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11161422-732b-41fa-a8ba-f8c1e4f9217a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:05.816722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11161422-732b-41fa-a8ba-f8c1e4f9217a admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.816722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11161422-732b-41fa-a8ba-f8c1e4f9217a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.816722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11161422-732b-41fa-a8ba-f8c1e4f9217a admin admin] PATH_INFO: `/v3/projects/137e8135b0844660a4da9e9ea847c83c/users/16eb7652b1aa43658e9518dc5c9eae9f/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.823760 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1a8b5-60c3-427d-a084-acb54f55a564 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:05.825584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1fe1a8b5-60c3-427d-a084-acb54f55a564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:05.828143 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11161422-732b-41fa-a8ba-f8c1e4f9217a admin admin] RBAC: Authorizing `identity:create_grant(project_id=137e8135b0844660a4da9e9ea847c83c, user_id=16eb7652b1aa43658e9518dc5c9eae9f, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:05.830881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11161422-732b-41fa-a8ba-f8c1e4f9217a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:05.844992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1214/2435] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:05 2022] PUT /identity/v3/projects/137e8135b0844660a4da9e9ea847c83c/users/16eb7652b1aa43658e9518dc5c9eae9f/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:05.856495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496dbf1-cead-4260-b5e8-dcd8069ee607 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.856952 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496dbf1-cead-4260-b5e8-dcd8069ee607 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.857227 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496dbf1-cead-4260-b5e8-dcd8069ee607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.884723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0496dbf1-cead-4260-b5e8-dcd8069ee607 None None] MFA Rules not processed for user `16eb7652b1aa43658e9518dc5c9eae9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:05.885696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1a8b5-60c3-427d-a084-acb54f55a564 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:05.886182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0496dbf1-cead-4260-b5e8-dcd8069ee607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:05.897522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:05.935674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9baf07-f288-462b-8072-345925ce7433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:05.936291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9baf07-f288-462b-8072-345925ce7433 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:05.936586 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9baf07-f288-462b-8072-345925ce7433 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:05.936836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9baf07-f288-462b-8072-345925ce7433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:05.938655 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-9d9baf07-f288-462b-8072-345925ce7433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:05.949900 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1215/2436] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:05.954175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496dbf1-cead-4260-b5e8-dcd8069ee607 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:05.962175 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:06.003498 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9baf07-f288-462b-8072-345925ce7433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:06.005219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9baf07-f288-462b-8072-345925ce7433 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:06.015113 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1222/2437] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:38:06.016589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496dbf1-cead-4260-b5e8-dcd8069ee607 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:06.021249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3f66d-0df1-4bfa-b94c-850cf602ad79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:06.021863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3f66d-0df1-4bfa-b94c-850cf602ad79 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:06.022138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3f66d-0df1-4bfa-b94c-850cf602ad79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:06.022405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3f66d-0df1-4bfa-b94c-850cf602ad79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:06.023112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3f66d-0df1-4bfa-b94c-850cf602ad79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:06.025015 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3f66d-0df1-4bfa-b94c-850cf602ad79 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:06.053216 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1216/2438] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:05 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:06.057398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9baf07-f288-462b-8072-345925ce7433 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:06.066871 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9baf07-f288-462b-8072-345925ce7433 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:06.132753 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d00ee-2b3e-491f-8fd2-9f9d1b2bec90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:06.133382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d00ee-2b3e-491f-8fd2-9f9d1b2bec90 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:06.133665 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d00ee-2b3e-491f-8fd2-9f9d1b2bec90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:06.133937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d00ee-2b3e-491f-8fd2-9f9d1b2bec90 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:06.134604 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d00ee-2b3e-491f-8fd2-9f9d1b2bec90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:06.136517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d00ee-2b3e-491f-8fd2-9f9d1b2bec90 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:06.162446 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1223/2439] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:06 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:06.163837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3f66d-0df1-4bfa-b94c-850cf602ad79 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:06.175394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3f66d-0df1-4bfa-b94c-850cf602ad79 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:06.181160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5458e-70be-4acb-b2dd-21bb3b037f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:06.181800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5458e-70be-4acb-b2dd-21bb3b037f6f admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:06.182153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5458e-70be-4acb-b2dd-21bb3b037f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:06.182444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5458e-70be-4acb-b2dd-21bb3b037f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:06.183146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5458e-70be-4acb-b2dd-21bb3b037f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:06.186584 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5458e-70be-4acb-b2dd-21bb3b037f6f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:06.197176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1217/2440] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:38:06.201549 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d00ee-2b3e-491f-8fd2-9f9d1b2bec90 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:06.211989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d00ee-2b3e-491f-8fd2-9f9d1b2bec90 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:06.217385 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197dbc6-4092-44a8-bbd5-91358476cac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:06.218122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197dbc6-4092-44a8-bbd5-91358476cac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:06.218597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197dbc6-4092-44a8-bbd5-91358476cac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:06.218738 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197dbc6-4092-44a8-bbd5-91358476cac3 admin admin] PATH_INFO: `/v3/projects/d320b71bbdbb450a860bc01549d043c3/users/05b9e207eeb2421a8715f23a2c9f482e/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:06.234433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197dbc6-4092-44a8-bbd5-91358476cac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d320b71bbdbb450a860bc01549d043c3, user_id=05b9e207eeb2421a8715f23a2c9f482e, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:06.237142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197dbc6-4092-44a8-bbd5-91358476cac3 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:06.252116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1224/2441] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:06 2022] PUT /identity/v3/projects/d320b71bbdbb450a860bc01549d043c3/users/05b9e207eeb2421a8715f23a2c9f482e/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:06.256170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5458e-70be-4acb-b2dd-21bb3b037f6f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:06.266783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5458e-70be-4acb-b2dd-21bb3b037f6f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:06.324319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa4922-173e-4ee8-b50a-33320671ba06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:06.324938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa4922-173e-4ee8-b50a-33320671ba06 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:06.325219 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa4922-173e-4ee8-b50a-33320671ba06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:06.325491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa4922-173e-4ee8-b50a-33320671ba06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:06.326134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa4922-173e-4ee8-b50a-33320671ba06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:06.327797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa4922-173e-4ee8-b50a-33320671ba06 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:06.338000 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1218/2442] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:06.342318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197dbc6-4092-44a8-bbd5-91358476cac3 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:06.352777 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197dbc6-4092-44a8-bbd5-91358476cac3 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:06.357693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567c467-3d55-450a-a2f7-22cf2b961f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:06.358354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567c467-3d55-450a-a2f7-22cf2b961f61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:06.358661 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567c467-3d55-450a-a2f7-22cf2b961f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:06.358941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567c467-3d55-450a-a2f7-22cf2b961f61 admin admin] PATH_INFO: `/v3/projects/d320b71bbdbb450a860bc01549d043c3/users/05b9e207eeb2421a8715f23a2c9f482e/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:06.360939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567c467-3d55-450a-a2f7-22cf2b961f61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d320b71bbdbb450a860bc01549d043c3, user_id=05b9e207eeb2421a8715f23a2c9f482e, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:06.363666 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567c467-3d55-450a-a2f7-22cf2b961f61 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:06.374287 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1225/2443] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:06 2022] PUT /identity/v3/projects/d320b71bbdbb450a860bc01549d043c3/users/05b9e207eeb2421a8715f23a2c9f482e/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:06.385209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71b2b0-4f4c-4628-b8d8-747c541b8a0b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:06.386150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71b2b0-4f4c-4628-b8d8-747c541b8a0b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:06.386440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71b2b0-4f4c-4628-b8d8-747c541b8a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:06.415074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-df71b2b0-4f4c-4628-b8d8-747c541b8a0b None None] MFA Rules not processed for user `05b9e207eeb2421a8715f23a2c9f482e`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:06.416317 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-df71b2b0-4f4c-4628-b8d8-747c541b8a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:06.487821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1219/2444] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:06.499187 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567c467-3d55-450a-a2f7-22cf2b961f61 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:06.500899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e567c467-3d55-450a-a2f7-22cf2b961f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:06.522151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:06.562915 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567c467-3d55-450a-a2f7-22cf2b961f61 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:06.566858 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85443f0f-c02b-4fb2-84c7-1f01b47040dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:06.567488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85443f0f-c02b-4fb2-84c7-1f01b47040dd service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:06.567770 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85443f0f-c02b-4fb2-84c7-1f01b47040dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:06.568019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85443f0f-c02b-4fb2-84c7-1f01b47040dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:06.577946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85443f0f-c02b-4fb2-84c7-1f01b47040dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:06.579434 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85443f0f-c02b-4fb2-84c7-1f01b47040dd service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:06.589449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1226/2445] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:06.633369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71b2b0-4f4c-4628-b8d8-747c541b8a0b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:06.635330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-df71b2b0-4f4c-4628-b8d8-747c541b8a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:06.642957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:06.650104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71b2b0-4f4c-4628-b8d8-747c541b8a0b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:06.654735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bba13-4324-4a21-8755-08f20339fdd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:06.654968 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bba13-4324-4a21-8755-08f20339fdd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:06.655379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bba13-4324-4a21-8755-08f20339fdd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:06.655653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bba13-4324-4a21-8755-08f20339fdd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:06.716396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bba13-4324-4a21-8755-08f20339fdd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:06.717942 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bba13-4324-4a21-8755-08f20339fdd0 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:06.727496 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1220/2446] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:38:09.415494 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85443f0f-c02b-4fb2-84c7-1f01b47040dd service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:09.416326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-85443f0f-c02b-4fb2-84c7-1f01b47040dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:09.485183 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85443f0f-c02b-4fb2-84c7-1f01b47040dd service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:09.490104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c298c32-31b4-4a83-81dd-9d7e4553f6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:09.490772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c298c32-31b4-4a83-81dd-9d7e4553f6a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:09.491035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c298c32-31b4-4a83-81dd-9d7e4553f6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:09.491302 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c298c32-31b4-4a83-81dd-9d7e4553f6a8 admin admin] PATH_INFO: `/v3/users/d095ed814b9e47e08d6b5a4fd2901c43` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:09.492438 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c298c32-31b4-4a83-81dd-9d7e4553f6a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d095ed814b9e47e08d6b5a4fd2901c43)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:09.494360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c298c32-31b4-4a83-81dd-9d7e4553f6a8 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:09.544884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5c298c32-31b4-4a83-81dd-9d7e4553f6a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd095ed814b9e47e08d6b5a4fd2901c43'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:09.545535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5c298c32-31b4-4a83-81dd-9d7e4553f6a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd095ed814b9e47e08d6b5a4fd2901c43'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:09.546232 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5c298c32-31b4-4a83-81dd-9d7e4553f6a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd095ed814b9e47e08d6b5a4fd2901c43'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:09.553266 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5c298c32-31b4-4a83-81dd-9d7e4553f6a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd095ed814b9e47e08d6b5a4fd2901c43'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:09.567588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5c298c32-31b4-4a83-81dd-9d7e4553f6a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd095ed814b9e47e08d6b5a4fd2901c43'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:09.595486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1227/2447] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:09 2022] DELETE /identity/v3/users/d095ed814b9e47e08d6b5a4fd2901c43 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:09.803200 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bba13-4324-4a21-8755-08f20339fdd0 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:09.822702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bba13-4324-4a21-8755-08f20339fdd0 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:09.863395 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c298c32-31b4-4a83-81dd-9d7e4553f6a8 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:09.864848 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5c298c32-31b4-4a83-81dd-9d7e4553f6a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:09.908764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ca829-7c6c-43b9-892b-99ff897983f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:09.909902 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ca829-7c6c-43b9-892b-99ff897983f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:09.910392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ca829-7c6c-43b9-892b-99ff897983f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:09.910765 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ca829-7c6c-43b9-892b-99ff897983f3 admin admin] PATH_INFO: `/v3/projects/f90f2e0c002343d88ae25dc7bfb2fe10` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:09.912271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ca829-7c6c-43b9-892b-99ff897983f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90f2e0c002343d88ae25dc7bfb2fe10)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:09.915514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ca829-7c6c-43b9-892b-99ff897983f3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:09.931396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:09.962232 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c298c32-31b4-4a83-81dd-9d7e4553f6a8 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:09.967280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc0f54-8e5e-481e-a9c6-2a74b382bb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:09.968022 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc0f54-8e5e-481e-a9c6-2a74b382bb4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:09.968371 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc0f54-8e5e-481e-a9c6-2a74b382bb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:09.968706 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc0f54-8e5e-481e-a9c6-2a74b382bb4e admin admin] PATH_INFO: `/v3/users/3d7f9d9480ca4832b09348688d1ae0b4` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:09.969963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc0f54-8e5e-481e-a9c6-2a74b382bb4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d7f9d9480ca4832b09348688d1ae0b4)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:09.972157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc0f54-8e5e-481e-a9c6-2a74b382bb4e admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:09.992973 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-296ca829-7c6c-43b9-892b-99ff897983f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90f2e0c002343d88ae25dc7bfb2fe10', 'request_id': 'req-296ca829-7c6c-43b9-892b-99ff897983f3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:09.999069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-296ca829-7c6c-43b9-892b-99ff897983f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90f2e0c002343d88ae25dc7bfb2fe10', 'request_id': 'req-296ca829-7c6c-43b9-892b-99ff897983f3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.005357 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-296ca829-7c6c-43b9-892b-99ff897983f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90f2e0c002343d88ae25dc7bfb2fe10', 'request_id': 'req-296ca829-7c6c-43b9-892b-99ff897983f3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.021329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-296ca829-7c6c-43b9-892b-99ff897983f3 admin admin] The token cache is being invalidate because project f90f2e0c002343d88ae25dc7bfb2fe10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:10.021637 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-296ca829-7c6c-43b9-892b-99ff897983f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.022232 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-296ca829-7c6c-43b9-892b-99ff897983f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.024067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1221/2448] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:09 2022] DELETE /identity/v3/projects/f90f2e0c002343d88ae25dc7bfb2fe10 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:10.028996 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce761a7-e14e-471b-9ea9-24138145c463 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:10.029273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce761a7-e14e-471b-9ea9-24138145c463 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:10.029538 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce761a7-e14e-471b-9ea9-24138145c463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:10.032052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d8cc0f54-8e5e-481e-a9c6-2a74b382bb4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d7f9d9480ca4832b09348688d1ae0b4'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.032690 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d8cc0f54-8e5e-481e-a9c6-2a74b382bb4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d7f9d9480ca4832b09348688d1ae0b4'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.033290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d8cc0f54-8e5e-481e-a9c6-2a74b382bb4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d7f9d9480ca4832b09348688d1ae0b4'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.042161 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d8cc0f54-8e5e-481e-a9c6-2a74b382bb4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d7f9d9480ca4832b09348688d1ae0b4'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.051263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-cce761a7-e14e-471b-9ea9-24138145c463 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:10.052610 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cce761a7-e14e-471b-9ea9-24138145c463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:10.058932 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d8cc0f54-8e5e-481e-a9c6-2a74b382bb4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d7f9d9480ca4832b09348688d1ae0b4'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.078004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1228/2449] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:09 2022] DELETE /identity/v3/users/3d7f9d9480ca4832b09348688d1ae0b4 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:10.085025 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:10.086021 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66365f1-bae9-4076-9174-4d99c4aa25d3 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:10.086885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66365f1-bae9-4076-9174-4d99c4aa25d3 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:10.087172 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66365f1-bae9-4076-9174-4d99c4aa25d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:10.107798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f66365f1-bae9-4076-9174-4d99c4aa25d3 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:10.108995 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f66365f1-bae9-4076-9174-4d99c4aa25d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:10.117220 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1222/2450] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:10.126323 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37487cb-f151-40a4-81ee-bb202c5b9a3f None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:10.126652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37487cb-f151-40a4-81ee-bb202c5b9a3f None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:10.126926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37487cb-f151-40a4-81ee-bb202c5b9a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:10.145777 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e37487cb-f151-40a4-81ee-bb202c5b9a3f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:10.147009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e37487cb-f151-40a4-81ee-bb202c5b9a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:10.169242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1229/2451] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:10.180113 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66365f1-bae9-4076-9174-4d99c4aa25d3 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:10.181587 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f66365f1-bae9-4076-9174-4d99c4aa25d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:10.212273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1223/2452] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:10.216761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37487cb-f151-40a4-81ee-bb202c5b9a3f None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:10.226539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37487cb-f151-40a4-81ee-bb202c5b9a3f None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:10.231698 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5c50e-5ae3-4e9c-a2c4-bea1d7bbdc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:10.232313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5c50e-5ae3-4e9c-a2c4-bea1d7bbdc21 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:10.232590 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5c50e-5ae3-4e9c-a2c4-bea1d7bbdc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:10.232862 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5c50e-5ae3-4e9c-a2c4-bea1d7bbdc21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:10.233508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5c50e-5ae3-4e9c-a2c4-bea1d7bbdc21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:38:10.233811 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5c50e-5ae3-4e9c-a2c4-bea1d7bbdc21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:10.236103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5c50e-5ae3-4e9c-a2c4-bea1d7bbdc21 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:10.239336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66365f1-bae9-4076-9174-4d99c4aa25d3 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:10.243072 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101e2ee-dc8c-4426-b766-55261f2fff5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:10.243630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101e2ee-dc8c-4426-b766-55261f2fff5b service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:10.243907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101e2ee-dc8c-4426-b766-55261f2fff5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:10.244185 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101e2ee-dc8c-4426-b766-55261f2fff5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:10.247250 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1224/2453] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:38:10.251789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5c50e-5ae3-4e9c-a2c4-bea1d7bbdc21 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:10.255169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101e2ee-dc8c-4426-b766-55261f2fff5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:10.256645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101e2ee-dc8c-4426-b766-55261f2fff5b service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:10.261546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5c50e-5ae3-4e9c-a2c4-bea1d7bbdc21 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:10.266619 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5d688-4556-4a6c-bead-8df78e985081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:10.266787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1230/2454] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:38:10.267242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5d688-4556-4a6c-bead-8df78e985081 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:10.267518 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5d688-4556-4a6c-bead-8df78e985081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:10.267783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5d688-4556-4a6c-bead-8df78e985081 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:10.268400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5d688-4556-4a6c-bead-8df78e985081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:10.270342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5d688-4556-4a6c-bead-8df78e985081 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:10.302215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1225/2455] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:10 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:10.307157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101e2ee-dc8c-4426-b766-55261f2fff5b service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:10.312232 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:10.318373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101e2ee-dc8c-4426-b766-55261f2fff5b service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:10.382063 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4a2a2-27df-4224-84d9-6d1c2b094281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:10.382726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4a2a2-27df-4224-84d9-6d1c2b094281 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:10.383010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4a2a2-27df-4224-84d9-6d1c2b094281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:10.383308 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4a2a2-27df-4224-84d9-6d1c2b094281 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:10.384050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4a2a2-27df-4224-84d9-6d1c2b094281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:10.385948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4a2a2-27df-4224-84d9-6d1c2b094281 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:10.412337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1231/2456] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:10 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:10.417318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5d688-4556-4a6c-bead-8df78e985081 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:10.429196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5d688-4556-4a6c-bead-8df78e985081 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:10.435113 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46406e1c-0d5d-4031-89c5-663d57aef0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:10.435716 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46406e1c-0d5d-4031-89c5-663d57aef0fb admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:10.435989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46406e1c-0d5d-4031-89c5-663d57aef0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:10.436253 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46406e1c-0d5d-4031-89c5-663d57aef0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:10.436893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46406e1c-0d5d-4031-89c5-663d57aef0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:10.438537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46406e1c-0d5d-4031-89c5-663d57aef0fb admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:10.448845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1226/2457] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:10.452949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4a2a2-27df-4224-84d9-6d1c2b094281 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:10.463121 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4a2a2-27df-4224-84d9-6d1c2b094281 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:10.467756 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b9bd2-e5e3-4374-b611-cc7caa3f86e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:10.468356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b9bd2-e5e3-4374-b611-cc7caa3f86e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:10.468632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b9bd2-e5e3-4374-b611-cc7caa3f86e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:10.468899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b9bd2-e5e3-4374-b611-cc7caa3f86e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:10.469546 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b9bd2-e5e3-4374-b611-cc7caa3f86e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:10.471267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b9bd2-e5e3-4374-b611-cc7caa3f86e2 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:10.482825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1232/2458] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:10.485726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46406e1c-0d5d-4031-89c5-663d57aef0fb admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:10.496398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46406e1c-0d5d-4031-89c5-663d57aef0fb admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:10.501050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5ea68-f67c-49d9-b082-e6ddcb227637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:10.501734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5ea68-f67c-49d9-b082-e6ddcb227637 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:10.502014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5ea68-f67c-49d9-b082-e6ddcb227637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:10.502285 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5ea68-f67c-49d9-b082-e6ddcb227637 admin admin] PATH_INFO: `/v3/projects/e093d600ea0f484093caac4acb2c78d5/users/ddfaf643f3974784b7405975046c13d1/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:10.518207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5ea68-f67c-49d9-b082-e6ddcb227637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e093d600ea0f484093caac4acb2c78d5, user_id=ddfaf643f3974784b7405975046c13d1, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:10.520901 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5ea68-f67c-49d9-b082-e6ddcb227637 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:10.539997 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1227/2459] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:10 2022] PUT /identity/v3/projects/e093d600ea0f484093caac4acb2c78d5/users/ddfaf643f3974784b7405975046c13d1/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:10.553324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edf152-0c18-4739-83d7-a6d42cd57609 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:10.553614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edf152-0c18-4739-83d7-a6d42cd57609 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:10.553939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edf152-0c18-4739-83d7-a6d42cd57609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:10.555075 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5ea68-f67c-49d9-b082-e6ddcb227637 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:10.556970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dbc5ea68-f67c-49d9-b082-e6ddcb227637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:10.586413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-b1edf152-0c18-4739-83d7-a6d42cd57609 None None] MFA Rules not processed for user `ddfaf643f3974784b7405975046c13d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:10.587905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b1edf152-0c18-4739-83d7-a6d42cd57609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:10.628345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5ea68-f67c-49d9-b082-e6ddcb227637 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:10.632941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:10.633569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:10.633843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:10.634110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] PATH_INFO: `/v3/projects/dcb3a783282a45fcaaed58bf79841a3c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:10.635314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcb3a783282a45fcaaed58bf79841a3c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:10.637241 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:10.639862 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:10.656115 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1233/2460] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:10.668293 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edf152-0c18-4739-83d7-a6d42cd57609 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:10.669999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b1edf152-0c18-4739-83d7-a6d42cd57609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:10.686993 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcb3a783282a45fcaaed58bf79841a3c', 'request_id': 'req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.693769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcb3a783282a45fcaaed58bf79841a3c', 'request_id': 'req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.699851 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcb3a783282a45fcaaed58bf79841a3c', 'request_id': 'req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.713972 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] The token cache is being invalidate because project dcb3a783282a45fcaaed58bf79841a3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:10.714262 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.714908 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.716765 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1228/2461] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:10 2022] DELETE /identity/v3/projects/dcb3a783282a45fcaaed58bf79841a3c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:10.720500 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:10.721816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:10.745846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edf152-0c18-4739-83d7-a6d42cd57609 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:10.750313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:10.786686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d83ad1-ac78-4cb0-8dc4-e2c63e2c8d1f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:10.792549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525512c0-c597-4ba3-902e-c139116bb251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:10.793211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525512c0-c597-4ba3-902e-c139116bb251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:10.793489 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525512c0-c597-4ba3-902e-c139116bb251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:10.793851 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525512c0-c597-4ba3-902e-c139116bb251 admin admin] PATH_INFO: `/v3/users/16eb7652b1aa43658e9518dc5c9eae9f` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:10.795052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525512c0-c597-4ba3-902e-c139116bb251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16eb7652b1aa43658e9518dc5c9eae9f)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:10.796977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525512c0-c597-4ba3-902e-c139116bb251 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:10.811008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6d646-3ca8-4684-9deb-d48936796038 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:10.811615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d646-3ca8-4684-9deb-d48936796038 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:10.811891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d646-3ca8-4684-9deb-d48936796038 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:10.812165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d646-3ca8-4684-9deb-d48936796038 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:10.813999 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c3c6d646-3ca8-4684-9deb-d48936796038 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:10.846188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-525512c0-c597-4ba3-902e-c139116bb251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16eb7652b1aa43658e9518dc5c9eae9f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.846869 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-525512c0-c597-4ba3-902e-c139116bb251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16eb7652b1aa43658e9518dc5c9eae9f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.847496 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-525512c0-c597-4ba3-902e-c139116bb251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16eb7652b1aa43658e9518dc5c9eae9f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.854231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-525512c0-c597-4ba3-902e-c139116bb251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16eb7652b1aa43658e9518dc5c9eae9f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.865726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-525512c0-c597-4ba3-902e-c139116bb251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16eb7652b1aa43658e9518dc5c9eae9f'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:10.874146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6d646-3ca8-4684-9deb-d48936796038 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:10.875804 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6d646-3ca8-4684-9deb-d48936796038 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:10.881090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1229/2462] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:10 2022] DELETE /identity/v3/users/16eb7652b1aa43658e9518dc5c9eae9f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:10.885217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1234/2463] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:11.255651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525512c0-c597-4ba3-902e-c139116bb251 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:11.257249 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-525512c0-c597-4ba3-902e-c139116bb251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:11.277192 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:11.468070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525512c0-c597-4ba3-902e-c139116bb251 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:11.473159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:11.473784 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:11.474070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:11.474384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] PATH_INFO: `/v3/projects/137e8135b0844660a4da9e9ea847c83c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:11.475741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=137e8135b0844660a4da9e9ea847c83c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:11.477996 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:11.534854 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '137e8135b0844660a4da9e9ea847c83c', 'request_id': 'req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:11.542722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '137e8135b0844660a4da9e9ea847c83c', 'request_id': 'req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:11.548624 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '137e8135b0844660a4da9e9ea847c83c', 'request_id': 'req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:11.565385 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] The token cache is being invalidate because project 137e8135b0844660a4da9e9ea847c83c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:11.565705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:11.566858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:11.569200 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1230/2464] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:11 2022] DELETE /identity/v3/projects/137e8135b0844660a4da9e9ea847c83c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:11.573621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6d646-3ca8-4684-9deb-d48936796038 service nova] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:11.584615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6d646-3ca8-4684-9deb-d48936796038 service nova] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:11.640731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87aeac-2c0b-44e3-b87c-11e138ff8849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:11.641393 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87aeac-2c0b-44e3-b87c-11e138ff8849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:11.641615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87aeac-2c0b-44e3-b87c-11e138ff8849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:11.641898 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87aeac-2c0b-44e3-b87c-11e138ff8849 admin admin] PATH_INFO: `/v3/users/05b9e207eeb2421a8715f23a2c9f482e` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:11.643042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87aeac-2c0b-44e3-b87c-11e138ff8849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b9e207eeb2421a8715f23a2c9f482e)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:11.644956 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87aeac-2c0b-44e3-b87c-11e138ff8849 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:11.697898 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ac87aeac-2c0b-44e3-b87c-11e138ff8849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b9e207eeb2421a8715f23a2c9f482e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:11.698580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ac87aeac-2c0b-44e3-b87c-11e138ff8849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b9e207eeb2421a8715f23a2c9f482e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:11.699194 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ac87aeac-2c0b-44e3-b87c-11e138ff8849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b9e207eeb2421a8715f23a2c9f482e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:11.700375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:11.707412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ac87aeac-2c0b-44e3-b87c-11e138ff8849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b9e207eeb2421a8715f23a2c9f482e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:11.719096 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ac87aeac-2c0b-44e3-b87c-11e138ff8849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b9e207eeb2421a8715f23a2c9f482e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:11.733398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1235/2465] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:11 2022] DELETE /identity/v3/users/05b9e207eeb2421a8715f23a2c9f482e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:12.011223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:12.012916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:12.026341 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18388acf-8dc8-4ce8-b276-f61f46c5e9d5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:12.088431 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:12.089070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:12.089341 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:12.089603 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11 admin admin] PATH_INFO: `/v3/projects/d320b71bbdbb450a860bc01549d043c3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:12.090755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d320b71bbdbb450a860bc01549d043c3)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:12.092668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:12.136567 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:12.141031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd320b71bbdbb450a860bc01549d043c3', 'request_id': 'req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:12.148489 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd320b71bbdbb450a860bc01549d043c3', 'request_id': 'req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:12.154056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd320b71bbdbb450a860bc01549d043c3', 'request_id': 'req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:12.167831 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11 admin admin] The token cache is being invalidate because project d320b71bbdbb450a860bc01549d043c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:12.168117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:12.168766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f168e1a7-3c4e-497a-ad56-a5fda1e1ec11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:12.170284 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1231/2466] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:12 2022] DELETE /identity/v3/projects/d320b71bbdbb450a860bc01549d043c3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:12.176305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1c430-6da0-4f71-9f43-3fd0005dc407 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:12.176585 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1c430-6da0-4f71-9f43-3fd0005dc407 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:12.176851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1c430-6da0-4f71-9f43-3fd0005dc407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:12.197588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-8da1c430-6da0-4f71-9f43-3fd0005dc407 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:12.198855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8da1c430-6da0-4f71-9f43-3fd0005dc407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:12.262431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1236/2467] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:12.275202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e1dd4-aa49-4596-8fb3-a36ed75f0464 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:12.275660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e1dd4-aa49-4596-8fb3-a36ed75f0464 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:12.276028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e1dd4-aa49-4596-8fb3-a36ed75f0464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:12.300239 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-689e1dd4-aa49-4596-8fb3-a36ed75f0464 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:12.301413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-689e1dd4-aa49-4596-8fb3-a36ed75f0464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:12.310433 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1232/2468] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:12.315646 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1c430-6da0-4f71-9f43-3fd0005dc407 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:12.328022 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1c430-6da0-4f71-9f43-3fd0005dc407 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:12.334692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018edcd-ceae-4fb7-88b1-aea19b9bf730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:12.334692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018edcd-ceae-4fb7-88b1-aea19b9bf730 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:12.334692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018edcd-ceae-4fb7-88b1-aea19b9bf730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:12.334996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018edcd-ceae-4fb7-88b1-aea19b9bf730 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:12.335576 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018edcd-ceae-4fb7-88b1-aea19b9bf730 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:38:12.335971 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018edcd-ceae-4fb7-88b1-aea19b9bf730 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:12.337577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018edcd-ceae-4fb7-88b1-aea19b9bf730 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:12.350126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1237/2469] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:38:12.354827 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e1dd4-aa49-4596-8fb3-a36ed75f0464 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:12.366186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e1dd4-aa49-4596-8fb3-a36ed75f0464 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:12.370877 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43291b-e0f5-4d24-898f-2129c9f46e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:12.371479 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43291b-e0f5-4d24-898f-2129c9f46e03 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:12.371747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43291b-e0f5-4d24-898f-2129c9f46e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:12.372010 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43291b-e0f5-4d24-898f-2129c9f46e03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:12.372686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43291b-e0f5-4d24-898f-2129c9f46e03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:12.374734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43291b-e0f5-4d24-898f-2129c9f46e03 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:12.406812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1233/2470] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:12 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:12.411049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018edcd-ceae-4fb7-88b1-aea19b9bf730 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:12.421857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018edcd-ceae-4fb7-88b1-aea19b9bf730 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:12.493386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd988398-7815-4fa4-b29c-2ffcdbd5c92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:12.494068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd988398-7815-4fa4-b29c-2ffcdbd5c92e admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:12.494406 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd988398-7815-4fa4-b29c-2ffcdbd5c92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:12.494757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd988398-7815-4fa4-b29c-2ffcdbd5c92e admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:12.495455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd988398-7815-4fa4-b29c-2ffcdbd5c92e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:12.497437 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd988398-7815-4fa4-b29c-2ffcdbd5c92e admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:12.526781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1238/2471] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:12 2022] POST /identity/v3/users => generated 283 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:12.528556 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43291b-e0f5-4d24-898f-2129c9f46e03 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:12.541167 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43291b-e0f5-4d24-898f-2129c9f46e03 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:12.548970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dddee2-9dd0-4552-a642-36d7a6350206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:12.550019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dddee2-9dd0-4552-a642-36d7a6350206 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:12.550300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dddee2-9dd0-4552-a642-36d7a6350206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:12.550620 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dddee2-9dd0-4552-a642-36d7a6350206 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:12.551361 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dddee2-9dd0-4552-a642-36d7a6350206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:12.553449 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dddee2-9dd0-4552-a642-36d7a6350206 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:12.566611 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1234/2472] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:12.570223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd988398-7815-4fa4-b29c-2ffcdbd5c92e admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:12.581516 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd988398-7815-4fa4-b29c-2ffcdbd5c92e admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:12.586373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64299a35-6b51-441e-9562-157d6deae451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:12.587016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64299a35-6b51-441e-9562-157d6deae451 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:12.587323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64299a35-6b51-441e-9562-157d6deae451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:12.587592 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64299a35-6b51-441e-9562-157d6deae451 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:12.588235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64299a35-6b51-441e-9562-157d6deae451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:12.589831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64299a35-6b51-441e-9562-157d6deae451 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:12.606681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1239/2473] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:12.608427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dddee2-9dd0-4552-a642-36d7a6350206 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:12.618658 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dddee2-9dd0-4552-a642-36d7a6350206 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:12.623426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042301c-e552-4643-a331-48eb9c1fa3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:12.624078 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042301c-e552-4643-a331-48eb9c1fa3b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:12.624353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042301c-e552-4643-a331-48eb9c1fa3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:12.624623 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042301c-e552-4643-a331-48eb9c1fa3b1 admin admin] PATH_INFO: `/v3/projects/99c51c7a71be4e0ab1d022ce60378235/users/adb9a68a2a694bb29f4bf0bc0016a6fe/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:12.639885 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042301c-e552-4643-a331-48eb9c1fa3b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c51c7a71be4e0ab1d022ce60378235, user_id=adb9a68a2a694bb29f4bf0bc0016a6fe, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:12.642720 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042301c-e552-4643-a331-48eb9c1fa3b1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:12.657313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1235/2474] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:12 2022] PUT /identity/v3/projects/99c51c7a71be4e0ab1d022ce60378235/users/adb9a68a2a694bb29f4bf0bc0016a6fe/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:12.669967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8f764-0e11-4ef3-a8bb-1ecf9062b26c None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:12.670301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8f764-0e11-4ef3-a8bb-1ecf9062b26c None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:12.670603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8f764-0e11-4ef3-a8bb-1ecf9062b26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:12.698343 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-a8c8f764-0e11-4ef3-a8bb-1ecf9062b26c None None] MFA Rules not processed for user `adb9a68a2a694bb29f4bf0bc0016a6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:12.699580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a8c8f764-0e11-4ef3-a8bb-1ecf9062b26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:12.717354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:12.766189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1240/2475] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:12.770959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042301c-e552-4643-a331-48eb9c1fa3b1 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:12.844047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042301c-e552-4643-a331-48eb9c1fa3b1 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:12.849053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d30f2a-0100-476a-85c7-3de0871c06ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:12.849655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d30f2a-0100-476a-85c7-3de0871c06ea admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:12.849928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d30f2a-0100-476a-85c7-3de0871c06ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:12.850196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d30f2a-0100-476a-85c7-3de0871c06ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:12.850862 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d30f2a-0100-476a-85c7-3de0871c06ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:12.852761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d30f2a-0100-476a-85c7-3de0871c06ea admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:12.881535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1236/2476] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:12 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:12.885894 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8f764-0e11-4ef3-a8bb-1ecf9062b26c None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:12.896526 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8f764-0e11-4ef3-a8bb-1ecf9062b26c None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:12.954165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d57f33-d6de-489e-874a-46bd0bf6d53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:12.954823 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d57f33-d6de-489e-874a-46bd0bf6d53d admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:12.955102 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d57f33-d6de-489e-874a-46bd0bf6d53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:12.955351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d57f33-d6de-489e-874a-46bd0bf6d53d admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:12.955989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d57f33-d6de-489e-874a-46bd0bf6d53d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:12.957926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d57f33-d6de-489e-874a-46bd0bf6d53d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:12.984852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1241/2477] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:12 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:12.991095 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d30f2a-0100-476a-85c7-3de0871c06ea admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:13.002150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d30f2a-0100-476a-85c7-3de0871c06ea admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:13.007874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533c2fe-9eb5-422b-a5ad-3c645e7ecf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:13.008470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533c2fe-9eb5-422b-a5ad-3c645e7ecf8f admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:13.008746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533c2fe-9eb5-422b-a5ad-3c645e7ecf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:13.009011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533c2fe-9eb5-422b-a5ad-3c645e7ecf8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:13.009648 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533c2fe-9eb5-422b-a5ad-3c645e7ecf8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:13.011307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533c2fe-9eb5-422b-a5ad-3c645e7ecf8f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:13.021571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1237/2478] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:13.026070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d57f33-d6de-489e-874a-46bd0bf6d53d admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:13.040570 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d57f33-d6de-489e-874a-46bd0bf6d53d admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:13.045829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2ae69-6978-42f0-a53a-b291dcb75de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:13.046487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2ae69-6978-42f0-a53a-b291dcb75de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:13.046784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2ae69-6978-42f0-a53a-b291dcb75de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:13.047050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2ae69-6978-42f0-a53a-b291dcb75de8 admin admin] PATH_INFO: `/v3/projects/0f5fdb5ddfb04abea60edef07eef2fc7/users/f39197c9c0ca473ebdeb0f1c428ff3a9/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:13.062336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2ae69-6978-42f0-a53a-b291dcb75de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f5fdb5ddfb04abea60edef07eef2fc7, user_id=f39197c9c0ca473ebdeb0f1c428ff3a9, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:13.065057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2ae69-6978-42f0-a53a-b291dcb75de8 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:13.081205 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1242/2479] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:13 2022] PUT /identity/v3/projects/0f5fdb5ddfb04abea60edef07eef2fc7/users/f39197c9c0ca473ebdeb0f1c428ff3a9/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:13.085278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533c2fe-9eb5-422b-a5ad-3c645e7ecf8f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:13.095819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533c2fe-9eb5-422b-a5ad-3c645e7ecf8f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:13.137859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:13.155239 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b7602-7356-438d-a218-4ded9a9b401b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:13.155855 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b7602-7356-438d-a218-4ded9a9b401b admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:13.156130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b7602-7356-438d-a218-4ded9a9b401b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:13.156404 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b7602-7356-438d-a218-4ded9a9b401b admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:13.157032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b7602-7356-438d-a218-4ded9a9b401b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:13.158663 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b7602-7356-438d-a218-4ded9a9b401b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:13.168701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1238/2480] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:13.172909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2ae69-6978-42f0-a53a-b291dcb75de8 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:13.183134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2ae69-6978-42f0-a53a-b291dcb75de8 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:13.187575 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd9116-df19-4fdb-b544-7cc6c8bfcb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:13.188219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd9116-df19-4fdb-b544-7cc6c8bfcb3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:13.188672 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd9116-df19-4fdb-b544-7cc6c8bfcb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:13.188950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd9116-df19-4fdb-b544-7cc6c8bfcb3a admin admin] PATH_INFO: `/v3/projects/0f5fdb5ddfb04abea60edef07eef2fc7/users/f39197c9c0ca473ebdeb0f1c428ff3a9/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:13.190917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd9116-df19-4fdb-b544-7cc6c8bfcb3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f5fdb5ddfb04abea60edef07eef2fc7, user_id=f39197c9c0ca473ebdeb0f1c428ff3a9, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:13.193582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd9116-df19-4fdb-b544-7cc6c8bfcb3a admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:13.203846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1243/2481] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:13 2022] PUT /identity/v3/projects/0f5fdb5ddfb04abea60edef07eef2fc7/users/f39197c9c0ca473ebdeb0f1c428ff3a9/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:13.213971 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6f2f-a22a-4518-aeed-4ea5178db00a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:13.214252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6f2f-a22a-4518-aeed-4ea5178db00a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:13.214521 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6f2f-a22a-4518-aeed-4ea5178db00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:13.245738 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-29aa6f2f-a22a-4518-aeed-4ea5178db00a None None] MFA Rules not processed for user `f39197c9c0ca473ebdeb0f1c428ff3a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:13.246994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-29aa6f2f-a22a-4518-aeed-4ea5178db00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:13.322176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1239/2482] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:13.333799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd9116-df19-4fdb-b544-7cc6c8bfcb3a admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:13.335729 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-22dd9116-df19-4fdb-b544-7cc6c8bfcb3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:13.402377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd9116-df19-4fdb-b544-7cc6c8bfcb3a admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:13.406385 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42e916-bab0-44e7-bd7d-726d54224613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:13.407000 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42e916-bab0-44e7-bd7d-726d54224613 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:13.407270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42e916-bab0-44e7-bd7d-726d54224613 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:13.407534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42e916-bab0-44e7-bd7d-726d54224613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:13.417104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42e916-bab0-44e7-bd7d-726d54224613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:13.418883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42e916-bab0-44e7-bd7d-726d54224613 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:13.428358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1244/2483] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:13.515609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa6f2f-a22a-4518-aeed-4ea5178db00a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:13.525800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa6f2f-a22a-4518-aeed-4ea5178db00a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:13.529814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6a3d2-91c7-4800-8575-054e09b2f3f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:13.530458 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6a3d2-91c7-4800-8575-054e09b2f3f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:13.530708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6a3d2-91c7-4800-8575-054e09b2f3f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:13.530964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6a3d2-91c7-4800-8575-054e09b2f3f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:13.593464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6a3d2-91c7-4800-8575-054e09b2f3f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:13.594961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6a3d2-91c7-4800-8575-054e09b2f3f0 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:13.604821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1240/2484] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:18.286705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42e916-bab0-44e7-bd7d-726d54224613 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:18.288107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2a42e916-bab0-44e7-bd7d-726d54224613 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:18.310686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:18.388755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42e916-bab0-44e7-bd7d-726d54224613 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:18.393351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be3d3a-b199-4d99-8a14-0fef68644600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:18.393986 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be3d3a-b199-4d99-8a14-0fef68644600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:18.394265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be3d3a-b199-4d99-8a14-0fef68644600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:18.394568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be3d3a-b199-4d99-8a14-0fef68644600 admin admin] PATH_INFO: `/v3/users/ddfaf643f3974784b7405975046c13d1` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:18.395712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be3d3a-b199-4d99-8a14-0fef68644600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddfaf643f3974784b7405975046c13d1)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:18.397652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be3d3a-b199-4d99-8a14-0fef68644600 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:18.448578 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c0be3d3a-b199-4d99-8a14-0fef68644600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddfaf643f3974784b7405975046c13d1'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:18.449343 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c0be3d3a-b199-4d99-8a14-0fef68644600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddfaf643f3974784b7405975046c13d1'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:18.450202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c0be3d3a-b199-4d99-8a14-0fef68644600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddfaf643f3974784b7405975046c13d1'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:18.458054 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c0be3d3a-b199-4d99-8a14-0fef68644600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddfaf643f3974784b7405975046c13d1'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:18.470556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c0be3d3a-b199-4d99-8a14-0fef68644600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddfaf643f3974784b7405975046c13d1'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:18.483695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1245/2485] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:18 2022] DELETE /identity/v3/users/ddfaf643f3974784b7405975046c13d1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:18.488657 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5eda0b-c146-4503-b411-c26c5dcf4e6a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:18.489369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5eda0b-c146-4503-b411-c26c5dcf4e6a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:18.489369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5eda0b-c146-4503-b411-c26c5dcf4e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:18.508881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7e5eda0b-c146-4503-b411-c26c5dcf4e6a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:18.510218 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7e5eda0b-c146-4503-b411-c26c5dcf4e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:18.573843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1241/2486] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:18.583689 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be3d3a-b199-4d99-8a14-0fef68644600 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:18.585176 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-c0be3d3a-b199-4d99-8a14-0fef68644600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:18.661640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be3d3a-b199-4d99-8a14-0fef68644600 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:18.666102 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1de98-0b71-45b6-a8b9-54ba753a6ca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:18.666758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1de98-0b71-45b6-a8b9-54ba753a6ca0 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:18.667035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1de98-0b71-45b6-a8b9-54ba753a6ca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:18.667307 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1de98-0b71-45b6-a8b9-54ba753a6ca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:18.672427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:18.686962 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1de98-0b71-45b6-a8b9-54ba753a6ca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:18.688446 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1de98-0b71-45b6-a8b9-54ba753a6ca0 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:18.715924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1246/2487] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:38:19.001043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5eda0b-c146-4503-b411-c26c5dcf4e6a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:19.003702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7e5eda0b-c146-4503-b411-c26c5dcf4e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:19.007860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:19.066636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5eda0b-c146-4503-b411-c26c5dcf4e6a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:19.071986 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80df6d-6e85-4384-8e9f-86ee88425070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:19.072566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80df6d-6e85-4384-8e9f-86ee88425070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.072846 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80df6d-6e85-4384-8e9f-86ee88425070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.073107 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80df6d-6e85-4384-8e9f-86ee88425070 admin admin] PATH_INFO: `/v3/projects/e093d600ea0f484093caac4acb2c78d5` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:19.074264 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80df6d-6e85-4384-8e9f-86ee88425070 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e093d600ea0f484093caac4acb2c78d5)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:19.076224 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80df6d-6e85-4384-8e9f-86ee88425070 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:19.123119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bb80df6d-6e85-4384-8e9f-86ee88425070 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e093d600ea0f484093caac4acb2c78d5', 'request_id': 'req-bb80df6d-6e85-4384-8e9f-86ee88425070'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:19.129279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bb80df6d-6e85-4384-8e9f-86ee88425070 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e093d600ea0f484093caac4acb2c78d5', 'request_id': 'req-bb80df6d-6e85-4384-8e9f-86ee88425070'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:19.134971 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bb80df6d-6e85-4384-8e9f-86ee88425070 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e093d600ea0f484093caac4acb2c78d5', 'request_id': 'req-bb80df6d-6e85-4384-8e9f-86ee88425070'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:19.148151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bb80df6d-6e85-4384-8e9f-86ee88425070 admin admin] The token cache is being invalidate because project e093d600ea0f484093caac4acb2c78d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:19.148436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bb80df6d-6e85-4384-8e9f-86ee88425070 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:19.149114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bb80df6d-6e85-4384-8e9f-86ee88425070 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:19.150652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1242/2488] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:18 2022] DELETE /identity/v3/projects/e093d600ea0f484093caac4acb2c78d5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:19.155715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e1873-c36e-4338-a310-d004bdbda082 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.156029 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e1873-c36e-4338-a310-d004bdbda082 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.156301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e1873-c36e-4338-a310-d004bdbda082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:19.202989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-570e1873-c36e-4338-a310-d004bdbda082 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:19.204479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-570e1873-c36e-4338-a310-d004bdbda082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:19.266116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1247/2489] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:19.274114 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94898e42-574d-4e94-9753-1ff9e29d69ae None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.274445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94898e42-574d-4e94-9753-1ff9e29d69ae None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.274781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94898e42-574d-4e94-9753-1ff9e29d69ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:19.295649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-94898e42-574d-4e94-9753-1ff9e29d69ae None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:19.296850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-94898e42-574d-4e94-9753-1ff9e29d69ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:19.305292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1243/2490] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:19.309636 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e1873-c36e-4338-a310-d004bdbda082 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:19.319676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e1873-c36e-4338-a310-d004bdbda082 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:19.324282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a058b-66af-4625-b9f7-91848fa4388e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:19.324829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a058b-66af-4625-b9f7-91848fa4388e admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.325105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a058b-66af-4625-b9f7-91848fa4388e admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.325371 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a058b-66af-4625-b9f7-91848fa4388e admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:19.326020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a058b-66af-4625-b9f7-91848fa4388e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:38:19.326316 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a058b-66af-4625-b9f7-91848fa4388e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:19.327975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a058b-66af-4625-b9f7-91848fa4388e admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:19.339758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1248/2491] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:38:19.344983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94898e42-574d-4e94-9753-1ff9e29d69ae None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:19.356333 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94898e42-574d-4e94-9753-1ff9e29d69ae None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:19.361276 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04689d-0fe6-4416-9e40-1beb96e8e351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:19.361943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04689d-0fe6-4416-9e40-1beb96e8e351 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.362282 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04689d-0fe6-4416-9e40-1beb96e8e351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.362645 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04689d-0fe6-4416-9e40-1beb96e8e351 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:19.363362 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04689d-0fe6-4416-9e40-1beb96e8e351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:19.365332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04689d-0fe6-4416-9e40-1beb96e8e351 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:19.397286 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1244/2492] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:19 2022] POST /identity/v3/projects => generated 357 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:19.401811 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a058b-66af-4625-b9f7-91848fa4388e admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:19.411798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a058b-66af-4625-b9f7-91848fa4388e admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:19.471765 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86856e28-54ef-461e-a650-c7f268b3d75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:19.472449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86856e28-54ef-461e-a650-c7f268b3d75c admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.472784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86856e28-54ef-461e-a650-c7f268b3d75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.473071 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86856e28-54ef-461e-a650-c7f268b3d75c admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:19.473674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86856e28-54ef-461e-a650-c7f268b3d75c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:19.475656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86856e28-54ef-461e-a650-c7f268b3d75c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:19.505447 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1249/2493] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:19 2022] POST /identity/v3/users => generated 279 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:19.506791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04689d-0fe6-4416-9e40-1beb96e8e351 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:19.519771 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04689d-0fe6-4416-9e40-1beb96e8e351 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:19.525423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecd805-17fa-4b28-86f8-f4928e968803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:19.525619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86856e28-54ef-461e-a650-c7f268b3d75c admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:19.526397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecd805-17fa-4b28-86f8-f4928e968803 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.526723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecd805-17fa-4b28-86f8-f4928e968803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.527165 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecd805-17fa-4b28-86f8-f4928e968803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:19.527868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecd805-17fa-4b28-86f8-f4928e968803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:19.529538 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecd805-17fa-4b28-86f8-f4928e968803 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:19.530025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-86856e28-54ef-461e-a650-c7f268b3d75c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:19.542289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1245/2494] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:19.543412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86856e28-54ef-461e-a650-c7f268b3d75c admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:19.546240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecd805-17fa-4b28-86f8-f4928e968803 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:19.549913 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7616d-7574-4436-8432-ec6cd99b23c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:19.550762 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7616d-7574-4436-8432-ec6cd99b23c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.551062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7616d-7574-4436-8432-ec6cd99b23c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.551349 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7616d-7574-4436-8432-ec6cd99b23c0 admin admin] PATH_INFO: `/v3/users/adb9a68a2a694bb29f4bf0bc0016a6fe` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:19.553016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7616d-7574-4436-8432-ec6cd99b23c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adb9a68a2a694bb29f4bf0bc0016a6fe)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:19.554637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7616d-7574-4436-8432-ec6cd99b23c0 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:19.557427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecd805-17fa-4b28-86f8-f4928e968803 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:19.563509 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc701b-ebe4-41fd-9179-26b57a3ca679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:19.564234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc701b-ebe4-41fd-9179-26b57a3ca679 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.564601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc701b-ebe4-41fd-9179-26b57a3ca679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.564967 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc701b-ebe4-41fd-9179-26b57a3ca679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:19.565707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc701b-ebe4-41fd-9179-26b57a3ca679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:19.567966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc701b-ebe4-41fd-9179-26b57a3ca679 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:19.581910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1246/2495] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:38:19.586781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc701b-ebe4-41fd-9179-26b57a3ca679 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:19.598378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc701b-ebe4-41fd-9179-26b57a3ca679 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:19.600658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:19.605416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec21b5-04e8-4cb8-9300-0759c927c5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:19.606550 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec21b5-04e8-4cb8-9300-0759c927c5cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.606861 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec21b5-04e8-4cb8-9300-0759c927c5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.607136 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec21b5-04e8-4cb8-9300-0759c927c5cf admin admin] PATH_INFO: `/v3/projects/b3d501d666b9446f97b6a6186bcfba8b/users/7218a467fd61447691cfd129f92332d1/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:19.623989 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c1e7616d-7574-4436-8432-ec6cd99b23c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adb9a68a2a694bb29f4bf0bc0016a6fe'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:19.625370 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec21b5-04e8-4cb8-9300-0759c927c5cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3d501d666b9446f97b6a6186bcfba8b, user_id=7218a467fd61447691cfd129f92332d1, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:19.626333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c1e7616d-7574-4436-8432-ec6cd99b23c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adb9a68a2a694bb29f4bf0bc0016a6fe'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:19.629660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec21b5-04e8-4cb8-9300-0759c927c5cf admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:19.630920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c1e7616d-7574-4436-8432-ec6cd99b23c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adb9a68a2a694bb29f4bf0bc0016a6fe'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:19.640201 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c1e7616d-7574-4436-8432-ec6cd99b23c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adb9a68a2a694bb29f4bf0bc0016a6fe'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:19.646872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1247/2496] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:19 2022] PUT /identity/v3/projects/b3d501d666b9446f97b6a6186bcfba8b/users/7218a467fd61447691cfd129f92332d1/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:19.658722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695a399-6c54-403b-a7dd-772137c441ef None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.658997 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695a399-6c54-403b-a7dd-772137c441ef None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.659273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695a399-6c54-403b-a7dd-772137c441ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:19.668351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c1e7616d-7574-4436-8432-ec6cd99b23c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adb9a68a2a694bb29f4bf0bc0016a6fe'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:19.692639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3695a399-6c54-403b-a7dd-772137c441ef None None] MFA Rules not processed for user `7218a467fd61447691cfd129f92332d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:19.694081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3695a399-6c54-403b-a7dd-772137c441ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:19.698324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1250/2497] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:19 2022] DELETE /identity/v3/users/adb9a68a2a694bb29f4bf0bc0016a6fe => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:19.700699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96d040-945c-4588-bfec-b1311889e2ca None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.703579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96d040-945c-4588-bfec-b1311889e2ca None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.703863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96d040-945c-4588-bfec-b1311889e2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:19.735866 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:19.743630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-ad96d040-945c-4588-bfec-b1311889e2ca None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:19.744900 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ad96d040-945c-4588-bfec-b1311889e2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:19.784248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1248/2498] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:19 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:19.788562 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695a399-6c54-403b-a7dd-772137c441ef None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:19.791202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3695a399-6c54-403b-a7dd-772137c441ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:19.813946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1251/2499] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:19.826159 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96d040-945c-4588-bfec-b1311889e2ca None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:19.828081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ad96d040-945c-4588-bfec-b1311889e2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:19.862724 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695a399-6c54-403b-a7dd-772137c441ef None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:19.867930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed123b3-f274-45ac-b9c8-7a9267b15d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:19.868583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed123b3-f274-45ac-b9c8-7a9267b15d92 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.868859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed123b3-f274-45ac-b9c8-7a9267b15d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.869124 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed123b3-f274-45ac-b9c8-7a9267b15d92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:19.869756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed123b3-f274-45ac-b9c8-7a9267b15d92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:19.872648 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed123b3-f274-45ac-b9c8-7a9267b15d92 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:19.902665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1249/2500] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:19 2022] POST /identity/v3/projects => generated 355 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:19.905834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed123b3-f274-45ac-b9c8-7a9267b15d92 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:19.918807 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96d040-945c-4588-bfec-b1311889e2ca None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:19.968114 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:19.974036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed123b3-f274-45ac-b9c8-7a9267b15d92 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:19.981163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7fe07-c9a5-43dd-8c0b-64a50f25f600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:19.982028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c976933-a717-4033-905a-61bfb5156a6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:19.982767 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7fe07-c9a5-43dd-8c0b-64a50f25f600 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.983047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7fe07-c9a5-43dd-8c0b-64a50f25f600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.983313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7fe07-c9a5-43dd-8c0b-64a50f25f600 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:19.983944 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7fe07-c9a5-43dd-8c0b-64a50f25f600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:19.985902 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7fe07-c9a5-43dd-8c0b-64a50f25f600 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:19.988955 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c976933-a717-4033-905a-61bfb5156a6e service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:19.989232 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c976933-a717-4033-905a-61bfb5156a6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:19.989525 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c976933-a717-4033-905a-61bfb5156a6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:20.001476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c976933-a717-4033-905a-61bfb5156a6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:20.003046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c976933-a717-4033-905a-61bfb5156a6e service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:20.011039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1250/2501] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:19 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:20.015304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7fe07-c9a5-43dd-8c0b-64a50f25f600 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:20.017092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1252/2502] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:20.026367 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7fe07-c9a5-43dd-8c0b-64a50f25f600 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:20.032387 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81078342-4216-4c8f-9b70-6c60edf02b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:20.032762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81078342-4216-4c8f-9b70-6c60edf02b8a admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:20.033037 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81078342-4216-4c8f-9b70-6c60edf02b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:20.033303 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81078342-4216-4c8f-9b70-6c60edf02b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:20.034021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81078342-4216-4c8f-9b70-6c60edf02b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:20.035591 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81078342-4216-4c8f-9b70-6c60edf02b8a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:20.047275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1251/2503] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:38:20.052876 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c976933-a717-4033-905a-61bfb5156a6e service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:20.064514 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c976933-a717-4033-905a-61bfb5156a6e service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:20.069362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb00e8-39b2-4dee-84d3-fd6b8341eb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:20.070076 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb00e8-39b2-4dee-84d3-fd6b8341eb95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:20.070351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb00e8-39b2-4dee-84d3-fd6b8341eb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:20.070656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb00e8-39b2-4dee-84d3-fd6b8341eb95 admin admin] PATH_INFO: `/v3/projects/88a8b93c88af490084f6153c5c430419/users/6eb2cad831584fa08d8b928737e1bb3e/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:20.089044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb00e8-39b2-4dee-84d3-fd6b8341eb95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88a8b93c88af490084f6153c5c430419, user_id=6eb2cad831584fa08d8b928737e1bb3e, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:20.091961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb00e8-39b2-4dee-84d3-fd6b8341eb95 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:20.105823 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1253/2504] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:20 2022] PUT /identity/v3/projects/88a8b93c88af490084f6153c5c430419/users/6eb2cad831584fa08d8b928737e1bb3e/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:20.109781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81078342-4216-4c8f-9b70-6c60edf02b8a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:20.120635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81078342-4216-4c8f-9b70-6c60edf02b8a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:20.153746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:20.193977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581270e-a525-44b1-83d4-1ac4b6ef8f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:20.194705 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581270e-a525-44b1-83d4-1ac4b6ef8f9c admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:20.194986 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581270e-a525-44b1-83d4-1ac4b6ef8f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:20.195257 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581270e-a525-44b1-83d4-1ac4b6ef8f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:20.196123 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581270e-a525-44b1-83d4-1ac4b6ef8f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:20.197817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581270e-a525-44b1-83d4-1ac4b6ef8f9c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:20.210050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1252/2505] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:20.214824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb00e8-39b2-4dee-84d3-fd6b8341eb95 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:20.228660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb00e8-39b2-4dee-84d3-fd6b8341eb95 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:20.237950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e0eaa-bd0d-41ea-a0e9-f36de5a7246a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:20.237950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e0eaa-bd0d-41ea-a0e9-f36de5a7246a admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:20.237950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e0eaa-bd0d-41ea-a0e9-f36de5a7246a admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:20.237950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e0eaa-bd0d-41ea-a0e9-f36de5a7246a admin admin] PATH_INFO: `/v3/projects/88a8b93c88af490084f6153c5c430419/users/6eb2cad831584fa08d8b928737e1bb3e/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:20.239860 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e0eaa-bd0d-41ea-a0e9-f36de5a7246a admin admin] RBAC: Authorizing `identity:create_grant(project_id=88a8b93c88af490084f6153c5c430419, user_id=6eb2cad831584fa08d8b928737e1bb3e, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:20.242576 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e0eaa-bd0d-41ea-a0e9-f36de5a7246a admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:20.255175 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1254/2506] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:20 2022] PUT /identity/v3/projects/88a8b93c88af490084f6153c5c430419/users/6eb2cad831584fa08d8b928737e1bb3e/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:20.266906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed96b09-f6d9-4747-b705-bb76cbcdae2a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:20.267211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed96b09-f6d9-4747-b705-bb76cbcdae2a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:20.267470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed96b09-f6d9-4747-b705-bb76cbcdae2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:20.296759 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-eed96b09-f6d9-4747-b705-bb76cbcdae2a None None] MFA Rules not processed for user `6eb2cad831584fa08d8b928737e1bb3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:20.297987 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-eed96b09-f6d9-4747-b705-bb76cbcdae2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:20.381173 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1253/2507] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:20 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:20.390420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e0eaa-bd0d-41ea-a0e9-f36de5a7246a admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:20.392894 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-852e0eaa-bd0d-41ea-a0e9-f36de5a7246a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:20.518944 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e0eaa-bd0d-41ea-a0e9-f36de5a7246a admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:20.527818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed96b09-f6d9-4747-b705-bb76cbcdae2a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:20.529692 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-eed96b09-f6d9-4747-b705-bb76cbcdae2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:20.533425 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad499ef-67a6-4625-9dae-8fa0b75a21fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:20.534212 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad499ef-67a6-4625-9dae-8fa0b75a21fc service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:20.535528 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad499ef-67a6-4625-9dae-8fa0b75a21fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:20.536022 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad499ef-67a6-4625-9dae-8fa0b75a21fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:20.584686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad499ef-67a6-4625-9dae-8fa0b75a21fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:20.586408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad499ef-67a6-4625-9dae-8fa0b75a21fc service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:20.598589 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1255/2508] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:20 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:20.601702 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad499ef-67a6-4625-9dae-8fa0b75a21fc service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:20.603968 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-dad499ef-67a6-4625-9dae-8fa0b75a21fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:20.688451 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed96b09-f6d9-4747-b705-bb76cbcdae2a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:20.716751 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594c44a-b48b-421f-8386-9ce76980406c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:20.717374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594c44a-b48b-421f-8386-9ce76980406c service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:20.717652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594c44a-b48b-421f-8386-9ce76980406c service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:20.717921 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594c44a-b48b-421f-8386-9ce76980406c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:20.721431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad499ef-67a6-4625-9dae-8fa0b75a21fc service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:20.723113 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4594c44a-b48b-421f-8386-9ce76980406c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:20.738593 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594c44a-b48b-421f-8386-9ce76980406c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:20.740156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594c44a-b48b-421f-8386-9ce76980406c service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:20.753457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:20.754051 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:20.754327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:20.754621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] PATH_INFO: `/v3/projects/99c51c7a71be4e0ab1d022ce60378235` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:20.755801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c51c7a71be4e0ab1d022ce60378235)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:20.757739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:20.766675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1254/2509] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:20.770184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594c44a-b48b-421f-8386-9ce76980406c service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:20.795837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594c44a-b48b-421f-8386-9ce76980406c service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:20.804499 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cb432-aedd-4f0c-b8fe-e9829b1ccf77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:20.805125 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cb432-aedd-4f0c-b8fe-e9829b1ccf77 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:20.805528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cb432-aedd-4f0c-b8fe-e9829b1ccf77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:20.807161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cb432-aedd-4f0c-b8fe-e9829b1ccf77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:20.835151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:20.850026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c51c7a71be4e0ab1d022ce60378235', 'request_id': 'req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:20.856826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c51c7a71be4e0ab1d022ce60378235', 'request_id': 'req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:20.867748 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c51c7a71be4e0ab1d022ce60378235', 'request_id': 'req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:20.870636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:20.888759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] The token cache is being invalidate because project 99c51c7a71be4e0ab1d022ce60378235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:20.889662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:20.890442 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:20.893623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1256/2510] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:20 2022] DELETE /identity/v3/projects/99c51c7a71be4e0ab1d022ce60378235 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:20.897577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:20.900162 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:20.908233 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cb432-aedd-4f0c-b8fe-e9829b1ccf77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:20.909924 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cb432-aedd-4f0c-b8fe-e9829b1ccf77 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:20.925700 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1255/2511] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:20.974340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463f0d3-9421-4ec4-8f38-ba7428bc6b5d admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:20.981450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c6a58-0e21-48a6-abb8-a686de393425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:20.982100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c6a58-0e21-48a6-abb8-a686de393425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:20.982379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c6a58-0e21-48a6-abb8-a686de393425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:20.982703 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c6a58-0e21-48a6-abb8-a686de393425 admin admin] PATH_INFO: `/v3/users/f39197c9c0ca473ebdeb0f1c428ff3a9` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:20.983921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c6a58-0e21-48a6-abb8-a686de393425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f39197c9c0ca473ebdeb0f1c428ff3a9)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:20.986341 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c6a58-0e21-48a6-abb8-a686de393425 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:21.053150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c34c6a58-0e21-48a6-abb8-a686de393425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f39197c9c0ca473ebdeb0f1c428ff3a9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:21.055181 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c34c6a58-0e21-48a6-abb8-a686de393425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f39197c9c0ca473ebdeb0f1c428ff3a9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:21.060547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c34c6a58-0e21-48a6-abb8-a686de393425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f39197c9c0ca473ebdeb0f1c428ff3a9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:21.070429 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c34c6a58-0e21-48a6-abb8-a686de393425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f39197c9c0ca473ebdeb0f1c428ff3a9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:21.103062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c34c6a58-0e21-48a6-abb8-a686de393425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f39197c9c0ca473ebdeb0f1c428ff3a9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:21.124416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1257/2512] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:20 2022] DELETE /identity/v3/users/f39197c9c0ca473ebdeb0f1c428ff3a9 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:21.410313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cb432-aedd-4f0c-b8fe-e9829b1ccf77 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:21.423612 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cb432-aedd-4f0c-b8fe-e9829b1ccf77 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:21.485984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299abb4-36d6-401c-87c8-9d013be72d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:21.486676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299abb4-36d6-401c-87c8-9d013be72d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:21.486952 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299abb4-36d6-401c-87c8-9d013be72d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:21.487219 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299abb4-36d6-401c-87c8-9d013be72d04 admin admin] PATH_INFO: `/v3/projects/0f5fdb5ddfb04abea60edef07eef2fc7` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:21.488327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299abb4-36d6-401c-87c8-9d013be72d04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f5fdb5ddfb04abea60edef07eef2fc7)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:21.490237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299abb4-36d6-401c-87c8-9d013be72d04 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:21.545873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e299abb4-36d6-401c-87c8-9d013be72d04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f5fdb5ddfb04abea60edef07eef2fc7', 'request_id': 'req-e299abb4-36d6-401c-87c8-9d013be72d04'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:21.557093 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e299abb4-36d6-401c-87c8-9d013be72d04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f5fdb5ddfb04abea60edef07eef2fc7', 'request_id': 'req-e299abb4-36d6-401c-87c8-9d013be72d04'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:21.563118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e299abb4-36d6-401c-87c8-9d013be72d04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f5fdb5ddfb04abea60edef07eef2fc7', 'request_id': 'req-e299abb4-36d6-401c-87c8-9d013be72d04'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:21.581555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e299abb4-36d6-401c-87c8-9d013be72d04 admin admin] The token cache is being invalidate because project 0f5fdb5ddfb04abea60edef07eef2fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:21.582399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e299abb4-36d6-401c-87c8-9d013be72d04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:21.582606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-e299abb4-36d6-401c-87c8-9d013be72d04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:21.584041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1256/2513] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:21 2022] DELETE /identity/v3/projects/0f5fdb5ddfb04abea60edef07eef2fc7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:21.591212 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de57cd-b5b0-42ea-92d2-e6866c640dea None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:21.591801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de57cd-b5b0-42ea-92d2-e6866c640dea None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:21.591801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de57cd-b5b0-42ea-92d2-e6866c640dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:21.613500 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f7de57cd-b5b0-42ea-92d2-e6866c640dea None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:21.614908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f7de57cd-b5b0-42ea-92d2-e6866c640dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:21.669185 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:21.695551 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1258/2514] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:21.703543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2061a6-b0b3-4d17-af0a-0cbf65e324e1 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:21.703894 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2061a6-b0b3-4d17-af0a-0cbf65e324e1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:21.704377 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2061a6-b0b3-4d17-af0a-0cbf65e324e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:21.724949 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-4d2061a6-b0b3-4d17-af0a-0cbf65e324e1 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:21.726239 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4d2061a6-b0b3-4d17-af0a-0cbf65e324e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:21.739981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1257/2515] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:21.744440 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de57cd-b5b0-42ea-92d2-e6866c640dea None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:21.758293 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de57cd-b5b0-42ea-92d2-e6866c640dea None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:21.764539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5212a6-dd15-489f-9006-16c576116e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:21.765415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5212a6-dd15-489f-9006-16c576116e7f admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:21.765827 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5212a6-dd15-489f-9006-16c576116e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:21.766099 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5212a6-dd15-489f-9006-16c576116e7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:21.766889 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5212a6-dd15-489f-9006-16c576116e7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:38:21.767187 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5212a6-dd15-489f-9006-16c576116e7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:21.768915 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5212a6-dd15-489f-9006-16c576116e7f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:21.782338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1259/2516] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:38:21.786773 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2061a6-b0b3-4d17-af0a-0cbf65e324e1 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:21.798355 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2061a6-b0b3-4d17-af0a-0cbf65e324e1 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:21.803116 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be2164-b7a9-47ce-b60b-57ff13e1d65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:21.803760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be2164-b7a9-47ce-b60b-57ff13e1d65a admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:21.804044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be2164-b7a9-47ce-b60b-57ff13e1d65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:21.804308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be2164-b7a9-47ce-b60b-57ff13e1d65a admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:21.804942 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be2164-b7a9-47ce-b60b-57ff13e1d65a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:21.807005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be2164-b7a9-47ce-b60b-57ff13e1d65a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:21.839679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1258/2517] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:21 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:21.841250 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5212a6-dd15-489f-9006-16c576116e7f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:21.852164 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5212a6-dd15-489f-9006-16c576116e7f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:21.909842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff263a8-bf53-417e-9f4c-6ab868623adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:21.910447 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff263a8-bf53-417e-9f4c-6ab868623adc admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:21.910763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff263a8-bf53-417e-9f4c-6ab868623adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:21.911037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff263a8-bf53-417e-9f4c-6ab868623adc admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:21.911686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff263a8-bf53-417e-9f4c-6ab868623adc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:21.913623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff263a8-bf53-417e-9f4c-6ab868623adc admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:21.940638 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1260/2518] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:21 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:21.941913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be2164-b7a9-47ce-b60b-57ff13e1d65a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:21.952900 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be2164-b7a9-47ce-b60b-57ff13e1d65a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:21.958392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65bf45-4587-42b3-bb69-514b7f4c299b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:21.959154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65bf45-4587-42b3-bb69-514b7f4c299b admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:21.959430 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65bf45-4587-42b3-bb69-514b7f4c299b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:21.959684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65bf45-4587-42b3-bb69-514b7f4c299b admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:21.960323 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65bf45-4587-42b3-bb69-514b7f4c299b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:21.961926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65bf45-4587-42b3-bb69-514b7f4c299b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:21.973691 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1259/2519] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:21.978637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff263a8-bf53-417e-9f4c-6ab868623adc admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:21.991947 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff263a8-bf53-417e-9f4c-6ab868623adc admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:21.997583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ae3b2-0ad5-43a3-b2e5-f43c23c7243d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:21.998237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ae3b2-0ad5-43a3-b2e5-f43c23c7243d admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:21.998539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ae3b2-0ad5-43a3-b2e5-f43c23c7243d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:21.998813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ae3b2-0ad5-43a3-b2e5-f43c23c7243d admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:21.999505 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ae3b2-0ad5-43a3-b2e5-f43c23c7243d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:22.001280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ae3b2-0ad5-43a3-b2e5-f43c23c7243d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:22.013594 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1261/2520] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:22.017867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65bf45-4587-42b3-bb69-514b7f4c299b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:22.056188 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65bf45-4587-42b3-bb69-514b7f4c299b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:22.063401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f309ec-f4ac-48a4-8c7e-b2f37e4b1df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:22.064851 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f309ec-f4ac-48a4-8c7e-b2f37e4b1df4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:22.065157 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f309ec-f4ac-48a4-8c7e-b2f37e4b1df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:22.065399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f309ec-f4ac-48a4-8c7e-b2f37e4b1df4 admin admin] PATH_INFO: `/v3/projects/1c6c1e895b5a49009d67f5d7e0ab6058/users/5ae302ea931a42b5938884e79c2ae381/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:22.095357 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f309ec-f4ac-48a4-8c7e-b2f37e4b1df4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c6c1e895b5a49009d67f5d7e0ab6058, user_id=5ae302ea931a42b5938884e79c2ae381, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:22.098077 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f309ec-f4ac-48a4-8c7e-b2f37e4b1df4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:22.110790 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:22.113716 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1260/2521] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:22 2022] PUT /identity/v3/projects/1c6c1e895b5a49009d67f5d7e0ab6058/users/5ae302ea931a42b5938884e79c2ae381/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:22.119535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17851536-6fee-4f88-a72b-779d9dfe1a9b None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:22.119826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17851536-6fee-4f88-a72b-779d9dfe1a9b None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:22.120097 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17851536-6fee-4f88-a72b-779d9dfe1a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:22.155884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-17851536-6fee-4f88-a72b-779d9dfe1a9b None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:22.160417 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-17851536-6fee-4f88-a72b-779d9dfe1a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:22.246531 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1262/2522] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:22.258692 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f309ec-f4ac-48a4-8c7e-b2f37e4b1df4 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:22.260540 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c8f309ec-f4ac-48a4-8c7e-b2f37e4b1df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:22.338537 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f309ec-f4ac-48a4-8c7e-b2f37e4b1df4 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:22.342704 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b711a-23db-4ab8-b1bc-b0079b3a0104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:22.343398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b711a-23db-4ab8-b1bc-b0079b3a0104 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:22.343742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b711a-23db-4ab8-b1bc-b0079b3a0104 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:22.344085 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b711a-23db-4ab8-b1bc-b0079b3a0104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:22.353429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b711a-23db-4ab8-b1bc-b0079b3a0104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:22.356044 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b711a-23db-4ab8-b1bc-b0079b3a0104 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:22.364386 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1261/2523] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:23.248620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24ffd-4fbb-4942-8620-ef3535dc0a0a None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:23.249131 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24ffd-4fbb-4942-8620-ef3535dc0a0a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:23.249240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24ffd-4fbb-4942-8620-ef3535dc0a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:23.280061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f9d24ffd-4fbb-4942-8620-ef3535dc0a0a None None] MFA Rules not processed for user `5ae302ea931a42b5938884e79c2ae381`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:23.281832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f9d24ffd-4fbb-4942-8620-ef3535dc0a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:23.326067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:23.366556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1263/2524] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:23.375149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b711a-23db-4ab8-b1bc-b0079b3a0104 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:23.377092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-372b711a-23db-4ab8-b1bc-b0079b3a0104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:23.461059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b711a-23db-4ab8-b1bc-b0079b3a0104 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:23.466380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd4624-ba4a-4397-a1d1-2cca04663458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:23.467347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd4624-ba4a-4397-a1d1-2cca04663458 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:23.467818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd4624-ba4a-4397-a1d1-2cca04663458 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:23.468380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd4624-ba4a-4397-a1d1-2cca04663458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:23.480485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd4624-ba4a-4397-a1d1-2cca04663458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:23.482605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd4624-ba4a-4397-a1d1-2cca04663458 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:23.493952 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1262/2525] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:25.001120 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da178a-b050-432f-af93-c577c7cd6f61 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:25.001677 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da178a-b050-432f-af93-c577c7cd6f61 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:25.001677 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da178a-b050-432f-af93-c577c7cd6f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:25.025061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-45da178a-b050-432f-af93-c577c7cd6f61 None None] MFA Rules not processed for user `5ae302ea931a42b5938884e79c2ae381`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:25.026605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-45da178a-b050-432f-af93-c577c7cd6f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:25.033518 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1264/2526] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:26.388270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd4624-ba4a-4397-a1d1-2cca04663458 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:26.398365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd4624-ba4a-4397-a1d1-2cca04663458 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:26.402242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b39ac-bca2-4bac-8726-b186751dcd0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:26.402914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b39ac-bca2-4bac-8726-b186751dcd0f service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:26.403208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b39ac-bca2-4bac-8726-b186751dcd0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:26.403473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b39ac-bca2-4bac-8726-b186751dcd0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:26.405265 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bc0b39ac-bca2-4bac-8726-b186751dcd0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:26.417985 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b39ac-bca2-4bac-8726-b186751dcd0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:26.419452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b39ac-bca2-4bac-8726-b186751dcd0f service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:26.428867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1263/2527] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:28.340871 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025de35-1c6a-4828-9fa9-f36511e69f89 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:28.341810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025de35-1c6a-4828-9fa9-f36511e69f89 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:28.343074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025de35-1c6a-4828-9fa9-f36511e69f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:28.453818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-2025de35-1c6a-4828-9fa9-f36511e69f89 None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:28.458461 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2025de35-1c6a-4828-9fa9-f36511e69f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:28.558396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1265/2528] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:38:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:28.612626 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b39ac-bca2-4bac-8726-b186751dcd0f service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:28.614106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bc0b39ac-bca2-4bac-8726-b186751dcd0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:28.686968 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b39ac-bca2-4bac-8726-b186751dcd0f service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:28.701784 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76e2b2-ccb3-493d-84c4-439b8ae89c5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:28.703420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76e2b2-ccb3-493d-84c4-439b8ae89c5f service swift] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:28.704056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76e2b2-ccb3-493d-84c4-439b8ae89c5f service swift] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:28.704338 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76e2b2-ccb3-493d-84c4-439b8ae89c5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:28.722429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76e2b2-ccb3-493d-84c4-439b8ae89c5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:28.725251 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76e2b2-ccb3-493d-84c4-439b8ae89c5f service swift] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:28.738400 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1264/2529] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:38:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:38:29.485673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025de35-1c6a-4828-9fa9-f36511e69f89 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:29.487307 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2025de35-1c6a-4828-9fa9-f36511e69f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:29.711743 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025de35-1c6a-4828-9fa9-f36511e69f89 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:29.723128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e3e4e-1d0a-421d-a55a-8fd45e6710f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:29.724284 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e3e4e-1d0a-421d-a55a-8fd45e6710f9 service nova] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:29.724861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e3e4e-1d0a-421d-a55a-8fd45e6710f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:29.725378 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e3e4e-1d0a-421d-a55a-8fd45e6710f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:29.727559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-004e3e4e-1d0a-421d-a55a-8fd45e6710f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:29.748002 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e3e4e-1d0a-421d-a55a-8fd45e6710f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:29.750188 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e3e4e-1d0a-421d-a55a-8fd45e6710f9 service nova] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:29.768079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1266/2530] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:31.285037 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76e2b2-ccb3-493d-84c4-439b8ae89c5f service swift] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:31.309996 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76e2b2-ccb3-493d-84c4-439b8ae89c5f service swift] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:31.327837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c6cab-89ea-43a9-881f-35479441859e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:31.331343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c6cab-89ea-43a9-881f-35479441859e service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:31.331737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c6cab-89ea-43a9-881f-35479441859e service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:31.332312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c6cab-89ea-43a9-881f-35479441859e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:31.338416 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f32c6cab-89ea-43a9-881f-35479441859e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:31.380006 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:31.483129 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c6cab-89ea-43a9-881f-35479441859e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:31.484947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c6cab-89ea-43a9-881f-35479441859e service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:31.519730 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1265/2531] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:31 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:31.579958 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e3e4e-1d0a-421d-a55a-8fd45e6710f9 service nova] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:31.581650 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-004e3e4e-1d0a-421d-a55a-8fd45e6710f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:31.614344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e3e4e-1d0a-421d-a55a-8fd45e6710f9 service nova] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:31.624699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a3c28-61cc-4a42-b11a-805deff2f595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:31.625349 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a3c28-61cc-4a42-b11a-805deff2f595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:31.625622 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a3c28-61cc-4a42-b11a-805deff2f595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:31.630848 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a3c28-61cc-4a42-b11a-805deff2f595 admin admin] PATH_INFO: `/v3/users/53c0721dc18343b18993ff5af098841d` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:31.632587 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a3c28-61cc-4a42-b11a-805deff2f595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53c0721dc18343b18993ff5af098841d)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:31.635071 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a3c28-61cc-4a42-b11a-805deff2f595 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:31.749751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-371a3c28-61cc-4a42-b11a-805deff2f595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53c0721dc18343b18993ff5af098841d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:31.750988 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-371a3c28-61cc-4a42-b11a-805deff2f595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53c0721dc18343b18993ff5af098841d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:31.751738 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-371a3c28-61cc-4a42-b11a-805deff2f595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53c0721dc18343b18993ff5af098841d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:31.753542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:31.768011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-371a3c28-61cc-4a42-b11a-805deff2f595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53c0721dc18343b18993ff5af098841d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:31.790251 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-371a3c28-61cc-4a42-b11a-805deff2f595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53c0721dc18343b18993ff5af098841d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:31.811288 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1267/2532] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:31 2022] DELETE /identity/v3/users/53c0721dc18343b18993ff5af098841d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:32.119247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c6cab-89ea-43a9-881f-35479441859e service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:32.139456 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c6cab-89ea-43a9-881f-35479441859e service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:32.239906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d76901-4409-4b5a-8992-8784b172a727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:32.240484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d76901-4409-4b5a-8992-8784b172a727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:32.240766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d76901-4409-4b5a-8992-8784b172a727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:32.241387 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d76901-4409-4b5a-8992-8784b172a727 admin admin] PATH_INFO: `/v3/projects/e17967a941be4a04bbc180e2cf3a7a55` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:32.243234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d76901-4409-4b5a-8992-8784b172a727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e17967a941be4a04bbc180e2cf3a7a55)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:32.247570 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d76901-4409-4b5a-8992-8784b172a727 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:32.315954 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-56d76901-4409-4b5a-8992-8784b172a727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e17967a941be4a04bbc180e2cf3a7a55', 'request_id': 'req-56d76901-4409-4b5a-8992-8784b172a727'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:32.325137 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-56d76901-4409-4b5a-8992-8784b172a727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e17967a941be4a04bbc180e2cf3a7a55', 'request_id': 'req-56d76901-4409-4b5a-8992-8784b172a727'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:32.331363 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-56d76901-4409-4b5a-8992-8784b172a727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e17967a941be4a04bbc180e2cf3a7a55', 'request_id': 'req-56d76901-4409-4b5a-8992-8784b172a727'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:32.347754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-56d76901-4409-4b5a-8992-8784b172a727 admin admin] The token cache is being invalidate because project e17967a941be4a04bbc180e2cf3a7a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:32.348587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-56d76901-4409-4b5a-8992-8784b172a727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:32.349433 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-56d76901-4409-4b5a-8992-8784b172a727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:32.351641 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1266/2533] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:32 2022] DELETE /identity/v3/projects/e17967a941be4a04bbc180e2cf3a7a55 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:32.358584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcfba3-aac1-4295-ba43-cd2a9e0a2883 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:32.358584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcfba3-aac1-4295-ba43-cd2a9e0a2883 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:32.358584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcfba3-aac1-4295-ba43-cd2a9e0a2883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:32.380687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-40fcfba3-aac1-4295-ba43-cd2a9e0a2883 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:32.382490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-40fcfba3-aac1-4295-ba43-cd2a9e0a2883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:32.455571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1268/2534] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:32.464088 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf4028-f731-46ac-a273-bea051ce3402 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:32.464376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf4028-f731-46ac-a273-bea051ce3402 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:32.464678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf4028-f731-46ac-a273-bea051ce3402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:32.485434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-05cf4028-f731-46ac-a273-bea051ce3402 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:32.486834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-05cf4028-f731-46ac-a273-bea051ce3402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:32.494785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1267/2535] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:32.499424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcfba3-aac1-4295-ba43-cd2a9e0a2883 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:32.509453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcfba3-aac1-4295-ba43-cd2a9e0a2883 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:32.515903 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792af93b-904a-4965-ac24-fd66dc8645b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:32.515903 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792af93b-904a-4965-ac24-fd66dc8645b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:32.515903 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792af93b-904a-4965-ac24-fd66dc8645b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:32.515903 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792af93b-904a-4965-ac24-fd66dc8645b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:32.516328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792af93b-904a-4965-ac24-fd66dc8645b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:38:32.516328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792af93b-904a-4965-ac24-fd66dc8645b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:32.518069 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792af93b-904a-4965-ac24-fd66dc8645b5 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:32.531842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1269/2536] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:38:32.536365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf4028-f731-46ac-a273-bea051ce3402 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:32.548629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf4028-f731-46ac-a273-bea051ce3402 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:32.556336 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2b539-793b-4ae8-af1d-369507eeccc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:32.556971 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2b539-793b-4ae8-af1d-369507eeccc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:32.557242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2b539-793b-4ae8-af1d-369507eeccc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:32.557512 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2b539-793b-4ae8-af1d-369507eeccc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:32.558210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2b539-793b-4ae8-af1d-369507eeccc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:32.560270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2b539-793b-4ae8-af1d-369507eeccc6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:32.594998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1268/2537] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:32 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:32.599110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792af93b-904a-4965-ac24-fd66dc8645b5 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:32.612115 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792af93b-904a-4965-ac24-fd66dc8645b5 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:32.679574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0713afd-8008-427d-a077-099fb8dd102f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:32.680185 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0713afd-8008-427d-a077-099fb8dd102f admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:32.680635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0713afd-8008-427d-a077-099fb8dd102f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:32.681186 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0713afd-8008-427d-a077-099fb8dd102f admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:32.681954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0713afd-8008-427d-a077-099fb8dd102f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:32.684286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0713afd-8008-427d-a077-099fb8dd102f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:32.711571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1270/2538] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:32 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:32.712834 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2b539-793b-4ae8-af1d-369507eeccc6 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:32.725515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2b539-793b-4ae8-af1d-369507eeccc6 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:32.730308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c67ec2-b3f4-4714-ae39-b2ad9fdc09dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:32.730949 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c67ec2-b3f4-4714-ae39-b2ad9fdc09dc admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:32.731227 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c67ec2-b3f4-4714-ae39-b2ad9fdc09dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:32.731500 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c67ec2-b3f4-4714-ae39-b2ad9fdc09dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:32.732155 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c67ec2-b3f4-4714-ae39-b2ad9fdc09dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:32.733815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c67ec2-b3f4-4714-ae39-b2ad9fdc09dc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:32.747533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1269/2539] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:32.749116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0713afd-8008-427d-a077-099fb8dd102f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:32.761452 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0713afd-8008-427d-a077-099fb8dd102f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:32.766365 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd7bf1-4b41-40e0-a221-a922e9c842d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:32.767100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd7bf1-4b41-40e0-a221-a922e9c842d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:32.767444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd7bf1-4b41-40e0-a221-a922e9c842d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:32.767784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd7bf1-4b41-40e0-a221-a922e9c842d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:32.768499 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd7bf1-4b41-40e0-a221-a922e9c842d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:32.770182 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd7bf1-4b41-40e0-a221-a922e9c842d6 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:32.785019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1271/2540] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:32.789833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c67ec2-b3f4-4714-ae39-b2ad9fdc09dc admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:32.801151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c67ec2-b3f4-4714-ae39-b2ad9fdc09dc admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:32.809653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223c930-efad-421f-806f-607a33c598e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:32.811180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223c930-efad-421f-806f-607a33c598e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:32.811559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223c930-efad-421f-806f-607a33c598e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:32.811926 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223c930-efad-421f-806f-607a33c598e9 admin admin] PATH_INFO: `/v3/projects/8c822ef7bbd4410d8356e7bfa8ad6819/users/6abc725d10904e7d9ce399f03dd67c4f/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:32.824273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:32.838637 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223c930-efad-421f-806f-607a33c598e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c822ef7bbd4410d8356e7bfa8ad6819, user_id=6abc725d10904e7d9ce399f03dd67c4f, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:32.841566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223c930-efad-421f-806f-607a33c598e9 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:32.874439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1270/2541] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:32 2022] PUT /identity/v3/projects/8c822ef7bbd4410d8356e7bfa8ad6819/users/6abc725d10904e7d9ce399f03dd67c4f/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:32.880696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e2897-8760-4e42-b815-8a4fa6a0964f None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:32.881241 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e2897-8760-4e42-b815-8a4fa6a0964f None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:32.881644 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e2897-8760-4e42-b815-8a4fa6a0964f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:32.933669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-da0e2897-8760-4e42-b815-8a4fa6a0964f None None] MFA Rules not processed for user `6abc725d10904e7d9ce399f03dd67c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:32.935122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-da0e2897-8760-4e42-b815-8a4fa6a0964f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:32.967042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:33.034180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1272/2542] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:33.046627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223c930-efad-421f-806f-607a33c598e9 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:33.048550 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3223c930-efad-421f-806f-607a33c598e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:33.148131 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223c930-efad-421f-806f-607a33c598e9 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:33.151695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b9743-0a54-479a-8b9b-7f2b2208c5fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:33.152380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b9743-0a54-479a-8b9b-7f2b2208c5fb service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:33.152618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b9743-0a54-479a-8b9b-7f2b2208c5fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:33.152893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b9743-0a54-479a-8b9b-7f2b2208c5fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:33.162065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b9743-0a54-479a-8b9b-7f2b2208c5fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:33.164421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b9743-0a54-479a-8b9b-7f2b2208c5fb service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:33.185473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1271/2543] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:33.211731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e2897-8760-4e42-b815-8a4fa6a0964f None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:33.348499 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e2897-8760-4e42-b815-8a4fa6a0964f None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:33.371275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13191bea-5d7d-4497-b680-f57f4aba296b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:33.372159 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13191bea-5d7d-4497-b680-f57f4aba296b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:33.372965 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13191bea-5d7d-4497-b680-f57f4aba296b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:33.373628 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13191bea-5d7d-4497-b680-f57f4aba296b admin admin] PATH_INFO: `/v3/users/6abc725d10904e7d9ce399f03dd67c4f` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:33.377391 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13191bea-5d7d-4497-b680-f57f4aba296b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6abc725d10904e7d9ce399f03dd67c4f)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:33.381363 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13191bea-5d7d-4497-b680-f57f4aba296b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:33.478801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-13191bea-5d7d-4497-b680-f57f4aba296b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6abc725d10904e7d9ce399f03dd67c4f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:33.479967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-13191bea-5d7d-4497-b680-f57f4aba296b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6abc725d10904e7d9ce399f03dd67c4f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:33.480747 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-13191bea-5d7d-4497-b680-f57f4aba296b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6abc725d10904e7d9ce399f03dd67c4f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:33.495508 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-13191bea-5d7d-4497-b680-f57f4aba296b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6abc725d10904e7d9ce399f03dd67c4f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:33.515685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-13191bea-5d7d-4497-b680-f57f4aba296b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6abc725d10904e7d9ce399f03dd67c4f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:33.554240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1273/2544] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:33 2022] DELETE /identity/v3/users/6abc725d10904e7d9ce399f03dd67c4f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:33.560835 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f8117-84b8-4f78-a2ee-b170b1b7759b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:33.561353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f8117-84b8-4f78-a2ee-b170b1b7759b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:33.561829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f8117-84b8-4f78-a2ee-b170b1b7759b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:33.597243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-2b7f8117-84b8-4f78-a2ee-b170b1b7759b None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:33.633683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2b7f8117-84b8-4f78-a2ee-b170b1b7759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:33.713510 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1272/2545] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:33.724552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13191bea-5d7d-4497-b680-f57f4aba296b admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:33.726067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-13191bea-5d7d-4497-b680-f57f4aba296b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:33.778220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:33.800251 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13191bea-5d7d-4497-b680-f57f4aba296b admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:33.804771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1d786-788f-4b47-bbc5-eb136058f13a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:33.806295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1d786-788f-4b47-bbc5-eb136058f13a service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:33.806723 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1d786-788f-4b47-bbc5-eb136058f13a service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:33.807060 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1d786-788f-4b47-bbc5-eb136058f13a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:33.833019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1d786-788f-4b47-bbc5-eb136058f13a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:33.836535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1d786-788f-4b47-bbc5-eb136058f13a service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:33.847757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1274/2546] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:34.190199 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f8117-84b8-4f78-a2ee-b170b1b7759b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:34.192148 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2b7f8117-84b8-4f78-a2ee-b170b1b7759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:34.210214 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f8117-84b8-4f78-a2ee-b170b1b7759b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:34.239382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842db8d-5171-4f11-8e23-344abc94d9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:34.239382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842db8d-5171-4f11-8e23-344abc94d9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:34.239382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842db8d-5171-4f11-8e23-344abc94d9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:34.239382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842db8d-5171-4f11-8e23-344abc94d9c8 admin admin] PATH_INFO: `/v3/projects/8c822ef7bbd4410d8356e7bfa8ad6819` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:34.241088 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c842db8d-5171-4f11-8e23-344abc94d9c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c822ef7bbd4410d8356e7bfa8ad6819)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:34.243111 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c842db8d-5171-4f11-8e23-344abc94d9c8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:34.342019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c842db8d-5171-4f11-8e23-344abc94d9c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c822ef7bbd4410d8356e7bfa8ad6819', 'request_id': 'req-c842db8d-5171-4f11-8e23-344abc94d9c8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:34.354905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c842db8d-5171-4f11-8e23-344abc94d9c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c822ef7bbd4410d8356e7bfa8ad6819', 'request_id': 'req-c842db8d-5171-4f11-8e23-344abc94d9c8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:34.363407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c842db8d-5171-4f11-8e23-344abc94d9c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c822ef7bbd4410d8356e7bfa8ad6819', 'request_id': 'req-c842db8d-5171-4f11-8e23-344abc94d9c8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:34.382605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c842db8d-5171-4f11-8e23-344abc94d9c8 admin admin] The token cache is being invalidate because project 8c822ef7bbd4410d8356e7bfa8ad6819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:34.382906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c842db8d-5171-4f11-8e23-344abc94d9c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:34.383702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-c842db8d-5171-4f11-8e23-344abc94d9c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:34.385515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1273/2547] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:34 2022] DELETE /identity/v3/projects/8c822ef7bbd4410d8356e7bfa8ad6819 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:34.390682 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64542c8a-196a-478e-b794-cac0d712ee99 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:34.391113 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64542c8a-196a-478e-b794-cac0d712ee99 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:34.391797 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64542c8a-196a-478e-b794-cac0d712ee99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:34.421213 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-64542c8a-196a-478e-b794-cac0d712ee99 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:34.423216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-64542c8a-196a-478e-b794-cac0d712ee99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:34.545431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1275/2548] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:34.554449 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8498d-c87e-4f9c-9a38-fd29f46a81eb None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:34.555069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8498d-c87e-4f9c-9a38-fd29f46a81eb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:34.561729 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8498d-c87e-4f9c-9a38-fd29f46a81eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:34.592796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0ea8498d-c87e-4f9c-9a38-fd29f46a81eb None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:34.595033 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0ea8498d-c87e-4f9c-9a38-fd29f46a81eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:34.615901 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1274/2549] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:34.619940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64542c8a-196a-478e-b794-cac0d712ee99 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:34.640395 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64542c8a-196a-478e-b794-cac0d712ee99 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:34.645400 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee4bc5-6ced-498d-8f1a-8f483d58658f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:34.646025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee4bc5-6ced-498d-8f1a-8f483d58658f admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:34.646302 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee4bc5-6ced-498d-8f1a-8f483d58658f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:34.646599 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee4bc5-6ced-498d-8f1a-8f483d58658f admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:34.647846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee4bc5-6ced-498d-8f1a-8f483d58658f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:38:34.648153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee4bc5-6ced-498d-8f1a-8f483d58658f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:34.654122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee4bc5-6ced-498d-8f1a-8f483d58658f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:34.680945 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1276/2550] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:38:34.686943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8498d-c87e-4f9c-9a38-fd29f46a81eb None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:34.715116 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8498d-c87e-4f9c-9a38-fd29f46a81eb None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:34.721002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdec1ca-2a5f-4649-b99d-a74a05f8c397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:34.721732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdec1ca-2a5f-4649-b99d-a74a05f8c397 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:34.722107 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdec1ca-2a5f-4649-b99d-a74a05f8c397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:34.722481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdec1ca-2a5f-4649-b99d-a74a05f8c397 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:34.723325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdec1ca-2a5f-4649-b99d-a74a05f8c397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:34.725485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdec1ca-2a5f-4649-b99d-a74a05f8c397 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:34.743019 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:34.792621 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1275/2551] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:34 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:34.797059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee4bc5-6ced-498d-8f1a-8f483d58658f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:34.810230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee4bc5-6ced-498d-8f1a-8f483d58658f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:34.906415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921a289-800d-4713-a95f-66f90625ecec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:34.907179 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921a289-800d-4713-a95f-66f90625ecec admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:34.907523 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921a289-800d-4713-a95f-66f90625ecec admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:34.907856 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921a289-800d-4713-a95f-66f90625ecec admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:34.908564 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4921a289-800d-4713-a95f-66f90625ecec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:34.910568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4921a289-800d-4713-a95f-66f90625ecec admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:34.936712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1277/2552] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:34 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:34.940517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdec1ca-2a5f-4649-b99d-a74a05f8c397 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:34.952587 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdec1ca-2a5f-4649-b99d-a74a05f8c397 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:34.958491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde44ddf-f09d-4cfb-8afd-c225ef863144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:34.959283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde44ddf-f09d-4cfb-8afd-c225ef863144 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:34.959628 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde44ddf-f09d-4cfb-8afd-c225ef863144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:34.959967 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde44ddf-f09d-4cfb-8afd-c225ef863144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:34.960687 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde44ddf-f09d-4cfb-8afd-c225ef863144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:34.962407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde44ddf-f09d-4cfb-8afd-c225ef863144 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:34.975881 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1276/2553] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:38:34.980323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921a289-800d-4713-a95f-66f90625ecec admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:34.994818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921a289-800d-4713-a95f-66f90625ecec admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:34.999767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe6b74-a98c-47b2-9a6d-12a25e5241dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:35.000388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe6b74-a98c-47b2-9a6d-12a25e5241dd admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:35.000670 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe6b74-a98c-47b2-9a6d-12a25e5241dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:35.000944 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe6b74-a98c-47b2-9a6d-12a25e5241dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:35.001597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe6b74-a98c-47b2-9a6d-12a25e5241dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:35.003285 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe6b74-a98c-47b2-9a6d-12a25e5241dd admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:35.019956 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1278/2554] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:35.021675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde44ddf-f09d-4cfb-8afd-c225ef863144 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:35.033770 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde44ddf-f09d-4cfb-8afd-c225ef863144 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:35.038918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a0a76-ee9a-4245-8f66-ba3938264a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:35.039573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a0a76-ee9a-4245-8f66-ba3938264a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:35.039847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a0a76-ee9a-4245-8f66-ba3938264a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:35.040110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a0a76-ee9a-4245-8f66-ba3938264a6b admin admin] PATH_INFO: `/v3/projects/1a517202e8a547d79768d8b53b041570/users/1746a031c5f84f5790c0ad58c14f12bc/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:35.055750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a0a76-ee9a-4245-8f66-ba3938264a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a517202e8a547d79768d8b53b041570, user_id=1746a031c5f84f5790c0ad58c14f12bc, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:35.058482 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a0a76-ee9a-4245-8f66-ba3938264a6b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:35.075921 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1277/2555] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:35 2022] PUT /identity/v3/projects/1a517202e8a547d79768d8b53b041570/users/1746a031c5f84f5790c0ad58c14f12bc/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:35.086320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3cb87-81f9-42e2-ae43-85cbafeda4ab None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:35.087196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3cb87-81f9-42e2-ae43-85cbafeda4ab None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:35.087780 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3cb87-81f9-42e2-ae43-85cbafeda4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:35.121261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-01f3cb87-81f9-42e2-ae43-85cbafeda4ab None None] MFA Rules not processed for user `1746a031c5f84f5790c0ad58c14f12bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:35.123568 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-01f3cb87-81f9-42e2-ae43-85cbafeda4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:35.168288 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:35.202623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1279/2556] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:35.216340 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a0a76-ee9a-4245-8f66-ba3938264a6b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:35.218342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-996a0a76-ee9a-4245-8f66-ba3938264a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:35.289269 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a0a76-ee9a-4245-8f66-ba3938264a6b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:35.293349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba0750-2644-4741-bfd9-b80154fd64bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:35.294141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba0750-2644-4741-bfd9-b80154fd64bc service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:35.294413 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba0750-2644-4741-bfd9-b80154fd64bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:35.294710 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba0750-2644-4741-bfd9-b80154fd64bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:35.303626 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba0750-2644-4741-bfd9-b80154fd64bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:35.307029 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba0750-2644-4741-bfd9-b80154fd64bc service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:35.328714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1278/2557] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:35.338905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3cb87-81f9-42e2-ae43-85cbafeda4ab None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:35.435471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3cb87-81f9-42e2-ae43-85cbafeda4ab None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:35.440553 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ef8e1-85ea-4f5f-be4e-d8b5dc23b540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:35.441203 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ef8e1-85ea-4f5f-be4e-d8b5dc23b540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:35.441617 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ef8e1-85ea-4f5f-be4e-d8b5dc23b540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:35.441893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ef8e1-85ea-4f5f-be4e-d8b5dc23b540 admin admin] PATH_INFO: `/v3/users/1746a031c5f84f5790c0ad58c14f12bc` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:35.443073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ef8e1-85ea-4f5f-be4e-d8b5dc23b540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1746a031c5f84f5790c0ad58c14f12bc)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:35.445169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ef8e1-85ea-4f5f-be4e-d8b5dc23b540 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:35.508875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-252ef8e1-85ea-4f5f-be4e-d8b5dc23b540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1746a031c5f84f5790c0ad58c14f12bc'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:35.508875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-252ef8e1-85ea-4f5f-be4e-d8b5dc23b540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1746a031c5f84f5790c0ad58c14f12bc'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:35.509600 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-252ef8e1-85ea-4f5f-be4e-d8b5dc23b540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1746a031c5f84f5790c0ad58c14f12bc'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:35.522532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-252ef8e1-85ea-4f5f-be4e-d8b5dc23b540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1746a031c5f84f5790c0ad58c14f12bc'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:35.543766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-252ef8e1-85ea-4f5f-be4e-d8b5dc23b540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1746a031c5f84f5790c0ad58c14f12bc'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:35.567621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1280/2558] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:35 2022] DELETE /identity/v3/users/1746a031c5f84f5790c0ad58c14f12bc => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:35.569657 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f227fb-d0cf-4ba1-8a70-d8aa2f1bf89c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:35.570152 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f227fb-d0cf-4ba1-8a70-d8aa2f1bf89c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:35.570437 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f227fb-d0cf-4ba1-8a70-d8aa2f1bf89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:35.594617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-68f227fb-d0cf-4ba1-8a70-d8aa2f1bf89c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:35.595918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-68f227fb-d0cf-4ba1-8a70-d8aa2f1bf89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:35.674328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1279/2559] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:35.682838 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ef8e1-85ea-4f5f-be4e-d8b5dc23b540 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:35.685282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-252ef8e1-85ea-4f5f-be4e-d8b5dc23b540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:35.762410 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ef8e1-85ea-4f5f-be4e-d8b5dc23b540 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:35.769124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a4da1-bcad-4078-8078-c994b968763a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:35.770933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a4da1-bcad-4078-8078-c994b968763a service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:35.771234 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a4da1-bcad-4078-8078-c994b968763a service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:35.771500 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a4da1-bcad-4078-8078-c994b968763a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:35.787132 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a4da1-bcad-4078-8078-c994b968763a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:35.789448 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a4da1-bcad-4078-8078-c994b968763a service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:35.796780 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:35.817828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1281/2560] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:36.292800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f227fb-d0cf-4ba1-8a70-d8aa2f1bf89c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:36.294977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-68f227fb-d0cf-4ba1-8a70-d8aa2f1bf89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:36.308194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f227fb-d0cf-4ba1-8a70-d8aa2f1bf89c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:36.340421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda185ed-a642-4e92-bf7e-e1b41abfee73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:36.341806 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda185ed-a642-4e92-bf7e-e1b41abfee73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:36.342174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda185ed-a642-4e92-bf7e-e1b41abfee73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:36.342536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda185ed-a642-4e92-bf7e-e1b41abfee73 admin admin] PATH_INFO: `/v3/projects/1a517202e8a547d79768d8b53b041570` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:36.344089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda185ed-a642-4e92-bf7e-e1b41abfee73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a517202e8a547d79768d8b53b041570)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:36.346374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda185ed-a642-4e92-bf7e-e1b41abfee73 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:36.410463 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:36.412651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cda185ed-a642-4e92-bf7e-e1b41abfee73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a517202e8a547d79768d8b53b041570', 'request_id': 'req-cda185ed-a642-4e92-bf7e-e1b41abfee73'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:36.422980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cda185ed-a642-4e92-bf7e-e1b41abfee73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a517202e8a547d79768d8b53b041570', 'request_id': 'req-cda185ed-a642-4e92-bf7e-e1b41abfee73'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:36.438231 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cda185ed-a642-4e92-bf7e-e1b41abfee73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a517202e8a547d79768d8b53b041570', 'request_id': 'req-cda185ed-a642-4e92-bf7e-e1b41abfee73'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:36.457226 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cda185ed-a642-4e92-bf7e-e1b41abfee73 admin admin] The token cache is being invalidate because project 1a517202e8a547d79768d8b53b041570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:36.457745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cda185ed-a642-4e92-bf7e-e1b41abfee73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:36.458629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-cda185ed-a642-4e92-bf7e-e1b41abfee73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:36.460622 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1280/2561] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:36 2022] DELETE /identity/v3/projects/1a517202e8a547d79768d8b53b041570 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:36.467117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f7be-a5c4-4ef5-a6fc-b661fa6dd811 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:36.467474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f7be-a5c4-4ef5-a6fc-b661fa6dd811 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:36.467809 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f7be-a5c4-4ef5-a6fc-b661fa6dd811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:36.491148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d0e4f7be-a5c4-4ef5-a6fc-b661fa6dd811 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:36.492346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d0e4f7be-a5c4-4ef5-a6fc-b661fa6dd811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:36.574832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1282/2562] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:36.582688 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27819a47-fc32-428d-be49-3608adcdff04 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:36.583216 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27819a47-fc32-428d-be49-3608adcdff04 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:36.583442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27819a47-fc32-428d-be49-3608adcdff04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:36.608994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-27819a47-fc32-428d-be49-3608adcdff04 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:36.611375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-27819a47-fc32-428d-be49-3608adcdff04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:36.629280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4f7be-a5c4-4ef5-a6fc-b661fa6dd811 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:36.648240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1281/2563] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:36.651249 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4f7be-a5c4-4ef5-a6fc-b661fa6dd811 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:36.661174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8a969-eb47-4977-a542-f3223932f446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:36.663073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8a969-eb47-4977-a542-f3223932f446 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:36.663224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8a969-eb47-4977-a542-f3223932f446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:36.663792 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8a969-eb47-4977-a542-f3223932f446 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:36.664449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8a969-eb47-4977-a542-f3223932f446 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:38:36.664755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8a969-eb47-4977-a542-f3223932f446 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:36.666377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8a969-eb47-4977-a542-f3223932f446 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:36.683135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1283/2564] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:38:36.697333 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27819a47-fc32-428d-be49-3608adcdff04 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:36.722961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27819a47-fc32-428d-be49-3608adcdff04 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:36.743291 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd09ee-c2da-45dd-bdb9-d9a26bfafa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:36.744331 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd09ee-c2da-45dd-bdb9-d9a26bfafa0b admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:36.744771 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd09ee-c2da-45dd-bdb9-d9a26bfafa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:36.745234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd09ee-c2da-45dd-bdb9-d9a26bfafa0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:36.746371 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd09ee-c2da-45dd-bdb9-d9a26bfafa0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:36.748745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd09ee-c2da-45dd-bdb9-d9a26bfafa0b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:36.807952 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8a969-eb47-4977-a542-f3223932f446 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:36.810970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1282/2565] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:36 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:36.823487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8a969-eb47-4977-a542-f3223932f446 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:36.972472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97e753-e1fb-41f5-914f-13b9147b89f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:36.973104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97e753-e1fb-41f5-914f-13b9147b89f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:36.973554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97e753-e1fb-41f5-914f-13b9147b89f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:36.973848 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97e753-e1fb-41f5-914f-13b9147b89f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:36.974767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97e753-e1fb-41f5-914f-13b9147b89f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:36.977285 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97e753-e1fb-41f5-914f-13b9147b89f2 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:37.019443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1284/2566] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:36 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:37.021058 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd09ee-c2da-45dd-bdb9-d9a26bfafa0b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:37.039345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd09ee-c2da-45dd-bdb9-d9a26bfafa0b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:37.044824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592e02b-864c-4c5c-aa99-4619b835082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:37.045447 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592e02b-864c-4c5c-aa99-4619b835082f admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:37.045711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592e02b-864c-4c5c-aa99-4619b835082f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:37.045976 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592e02b-864c-4c5c-aa99-4619b835082f admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:37.046684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592e02b-864c-4c5c-aa99-4619b835082f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:37.048545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592e02b-864c-4c5c-aa99-4619b835082f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:37.062557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1283/2567] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:37.066745 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97e753-e1fb-41f5-914f-13b9147b89f2 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:37.089336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97e753-e1fb-41f5-914f-13b9147b89f2 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:37.103760 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc0dce-64ff-4301-a4a9-37a5618fde66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:37.104443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc0dce-64ff-4301-a4a9-37a5618fde66 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:37.104726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc0dce-64ff-4301-a4a9-37a5618fde66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:37.104995 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc0dce-64ff-4301-a4a9-37a5618fde66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:37.105652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc0dce-64ff-4301-a4a9-37a5618fde66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:37.107388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc0dce-64ff-4301-a4a9-37a5618fde66 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:37.125413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1285/2568] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:37.127977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592e02b-864c-4c5c-aa99-4619b835082f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:37.154869 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592e02b-864c-4c5c-aa99-4619b835082f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:37.154869 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7169c-d26d-48a7-9dd7-7648756749af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:37.154869 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7169c-d26d-48a7-9dd7-7648756749af admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:37.154869 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7169c-d26d-48a7-9dd7-7648756749af admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:37.155390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7169c-d26d-48a7-9dd7-7648756749af admin admin] PATH_INFO: `/v3/projects/531efc4f316f453fb01618e7a472f4b4/users/f0a6c8454bdb4f54a1a5dc38381e7e28/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:37.171943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7169c-d26d-48a7-9dd7-7648756749af admin admin] RBAC: Authorizing `identity:create_grant(project_id=531efc4f316f453fb01618e7a472f4b4, user_id=f0a6c8454bdb4f54a1a5dc38381e7e28, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:37.177239 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7169c-d26d-48a7-9dd7-7648756749af admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:37.197827 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1284/2569] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:37 2022] PUT /identity/v3/projects/531efc4f316f453fb01618e7a472f4b4/users/f0a6c8454bdb4f54a1a5dc38381e7e28/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:37.209461 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e5beb-5581-45ca-aa70-b67ac18fe9ce None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:37.209750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e5beb-5581-45ca-aa70-b67ac18fe9ce None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:37.210022 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e5beb-5581-45ca-aa70-b67ac18fe9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:37.252344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-0f7e5beb-5581-45ca-aa70-b67ac18fe9ce None None] MFA Rules not processed for user `f0a6c8454bdb4f54a1a5dc38381e7e28`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:37.253786 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0f7e5beb-5581-45ca-aa70-b67ac18fe9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:37.329948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:37.346616 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1286/2570] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:37.351310 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7169c-d26d-48a7-9dd7-7648756749af admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:37.420129 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7169c-d26d-48a7-9dd7-7648756749af admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:37.424927 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cc190-e295-4efd-b8ea-979f5fc519d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:37.425539 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cc190-e295-4efd-b8ea-979f5fc519d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:37.425813 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cc190-e295-4efd-b8ea-979f5fc519d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:37.426079 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cc190-e295-4efd-b8ea-979f5fc519d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:37.426746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cc190-e295-4efd-b8ea-979f5fc519d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:37.428693 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cc190-e295-4efd-b8ea-979f5fc519d6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:37.462316 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1285/2571] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:37 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:37.467278 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e5beb-5581-45ca-aa70-b67ac18fe9ce None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:37.477949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e5beb-5581-45ca-aa70-b67ac18fe9ce None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:37.547482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f1db7-0f65-411f-8b9e-b06b0dcf869b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:37.548028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f1db7-0f65-411f-8b9e-b06b0dcf869b admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:37.548338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f1db7-0f65-411f-8b9e-b06b0dcf869b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:37.548782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f1db7-0f65-411f-8b9e-b06b0dcf869b admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:37.549419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f1db7-0f65-411f-8b9e-b06b0dcf869b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:37.551927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f1db7-0f65-411f-8b9e-b06b0dcf869b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:37.575789 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1287/2572] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:37 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:37.579556 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cc190-e295-4efd-b8ea-979f5fc519d6 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:37.591746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cc190-e295-4efd-b8ea-979f5fc519d6 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:37.598401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695b6c1-b978-4dca-bff9-8af85c93fd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:37.599088 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695b6c1-b978-4dca-bff9-8af85c93fd64 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:37.599366 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695b6c1-b978-4dca-bff9-8af85c93fd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:37.599640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695b6c1-b978-4dca-bff9-8af85c93fd64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:37.600301 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695b6c1-b978-4dca-bff9-8af85c93fd64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:37.601907 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695b6c1-b978-4dca-bff9-8af85c93fd64 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:37.619402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1286/2573] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:37.620511 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f1db7-0f65-411f-8b9e-b06b0dcf869b admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:37.632141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f1db7-0f65-411f-8b9e-b06b0dcf869b admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:37.637071 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678ad26-2d69-48d9-8686-c288f6db8ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:37.637693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678ad26-2d69-48d9-8686-c288f6db8ccb admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:37.637969 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678ad26-2d69-48d9-8686-c288f6db8ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:37.638240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678ad26-2d69-48d9-8686-c288f6db8ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:37.638997 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678ad26-2d69-48d9-8686-c288f6db8ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:37.640729 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678ad26-2d69-48d9-8686-c288f6db8ccb admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:37.653354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1288/2574] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:38:37.658048 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695b6c1-b978-4dca-bff9-8af85c93fd64 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:37.677355 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695b6c1-b978-4dca-bff9-8af85c93fd64 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:37.696176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f1594-d39f-4e12-bc9d-6c6a378eabf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:37.697273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f1594-d39f-4e12-bc9d-6c6a378eabf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:37.699867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f1594-d39f-4e12-bc9d-6c6a378eabf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:37.700964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f1594-d39f-4e12-bc9d-6c6a378eabf6 admin admin] PATH_INFO: `/v3/projects/764af9d74f804d918a36c3f5ac851e4e/users/b7a997640f994c40a16f5a7cfd71eb07/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:37.720183 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f1594-d39f-4e12-bc9d-6c6a378eabf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=764af9d74f804d918a36c3f5ac851e4e, user_id=b7a997640f994c40a16f5a7cfd71eb07, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:37.723320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f1594-d39f-4e12-bc9d-6c6a378eabf6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:37.752342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1287/2575] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:37 2022] PUT /identity/v3/projects/764af9d74f804d918a36c3f5ac851e4e/users/b7a997640f994c40a16f5a7cfd71eb07/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:37.763018 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff153a-7625-4655-8205-b5583bdc4f48 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:37.763361 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff153a-7625-4655-8205-b5583bdc4f48 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:37.763644 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff153a-7625-4655-8205-b5583bdc4f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:37.822459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-13ff153a-7625-4655-8205-b5583bdc4f48 None None] MFA Rules not processed for user `b7a997640f994c40a16f5a7cfd71eb07`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:37.823931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-13ff153a-7625-4655-8205-b5583bdc4f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:37.912220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1289/2576] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:37.917348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f1594-d39f-4e12-bc9d-6c6a378eabf6 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:37.954046 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:38.032905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f1594-d39f-4e12-bc9d-6c6a378eabf6 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:38.038635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c36dd-cc0f-4f42-b3e6-968d65ed43c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:38.039264 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c36dd-cc0f-4f42-b3e6-968d65ed43c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:38.039535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c36dd-cc0f-4f42-b3e6-968d65ed43c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:38.042286 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c36dd-cc0f-4f42-b3e6-968d65ed43c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:38.047701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c36dd-cc0f-4f42-b3e6-968d65ed43c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:38.047701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c36dd-cc0f-4f42-b3e6-968d65ed43c5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:38.094145 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1288/2577] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:37 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:38.095130 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff153a-7625-4655-8205-b5583bdc4f48 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:38.107638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c36dd-cc0f-4f42-b3e6-968d65ed43c5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:38.114322 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b53c36dd-cc0f-4f42-b3e6-968d65ed43c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:38.119785 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff153a-7625-4655-8205-b5583bdc4f48 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:38.247144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c36dd-cc0f-4f42-b3e6-968d65ed43c5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:38.252386 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d946a-9b4d-4cac-b699-628e0e8af2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:38.253270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d946a-9b4d-4cac-b699-628e0e8af2fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:38.253695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d946a-9b4d-4cac-b699-628e0e8af2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:38.254128 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d946a-9b4d-4cac-b699-628e0e8af2fc admin admin] PATH_INFO: `/v3/users/7218a467fd61447691cfd129f92332d1` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:38.262535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd407a-10a6-46b1-86dc-b9c8c39c2802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:38.262535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd407a-10a6-46b1-86dc-b9c8c39c2802 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:38.262535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd407a-10a6-46b1-86dc-b9c8c39c2802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:38.263053 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd407a-10a6-46b1-86dc-b9c8c39c2802 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:38.263574 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d946a-9b4d-4cac-b699-628e0e8af2fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7218a467fd61447691cfd129f92332d1)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:38.263834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd407a-10a6-46b1-86dc-b9c8c39c2802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:38.265933 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d946a-9b4d-4cac-b699-628e0e8af2fc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:38.266740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd407a-10a6-46b1-86dc-b9c8c39c2802 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:38.278267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:38.299875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1290/2578] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:38 2022] POST /identity/v3/users => generated 295 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:38.301210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd407a-10a6-46b1-86dc-b9c8c39c2802 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:38.330631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd407a-10a6-46b1-86dc-b9c8c39c2802 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:38.336009 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5b194-df70-4f65-8585-2339ed6597f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:38.336979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5b194-df70-4f65-8585-2339ed6597f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:38.337325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5b194-df70-4f65-8585-2339ed6597f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:38.337666 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5b194-df70-4f65-8585-2339ed6597f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:38.338387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5b194-df70-4f65-8585-2339ed6597f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:38.340921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5b194-df70-4f65-8585-2339ed6597f3 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:38.352751 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6d2d946a-9b4d-4cac-b699-628e0e8af2fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7218a467fd61447691cfd129f92332d1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:38.353515 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6d2d946a-9b4d-4cac-b699-628e0e8af2fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7218a467fd61447691cfd129f92332d1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:38.354316 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6d2d946a-9b4d-4cac-b699-628e0e8af2fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7218a467fd61447691cfd129f92332d1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:38.365045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1291/2579] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:38.367503 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6d2d946a-9b4d-4cac-b699-628e0e8af2fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7218a467fd61447691cfd129f92332d1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:38.369190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5b194-df70-4f65-8585-2339ed6597f3 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:38.371173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cda5b194-df70-4f65-8585-2339ed6597f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:38.396931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5b194-df70-4f65-8585-2339ed6597f3 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:38.398542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6d2d946a-9b4d-4cac-b699-628e0e8af2fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7218a467fd61447691cfd129f92332d1'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:38.405794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087f997-c953-4c2a-a72d-2d0da3e1ee41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:38.406402 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087f997-c953-4c2a-a72d-2d0da3e1ee41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:38.406703 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087f997-c953-4c2a-a72d-2d0da3e1ee41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:38.406975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087f997-c953-4c2a-a72d-2d0da3e1ee41 admin admin] PATH_INFO: `/v3/projects/ce366276b9044814aa1d82a6dff5bf49/users/43bd9c7694e642f88963d9aef84a1f66/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:38.428218 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1289/2580] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:38 2022] DELETE /identity/v3/users/7218a467fd61447691cfd129f92332d1 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:38.432674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940971e7-028f-4496-b24a-ad67b297e40c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:38.432952 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940971e7-028f-4496-b24a-ad67b297e40c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:38.433227 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940971e7-028f-4496-b24a-ad67b297e40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:38.459601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-940971e7-028f-4496-b24a-ad67b297e40c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:38.460910 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-940971e7-028f-4496-b24a-ad67b297e40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:38.485698 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:38.544061 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1290/2581] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:38.557055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940971e7-028f-4496-b24a-ad67b297e40c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:38.560486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-940971e7-028f-4496-b24a-ad67b297e40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:38.593831 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087f997-c953-4c2a-a72d-2d0da3e1ee41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce366276b9044814aa1d82a6dff5bf49, user_id=43bd9c7694e642f88963d9aef84a1f66, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:38.596652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087f997-c953-4c2a-a72d-2d0da3e1ee41 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:38.620023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1292/2582] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:38 2022] PUT /identity/v3/projects/ce366276b9044814aa1d82a6dff5bf49/users/43bd9c7694e642f88963d9aef84a1f66/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:38.623966 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087f997-c953-4c2a-a72d-2d0da3e1ee41 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:38.659498 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940971e7-028f-4496-b24a-ad67b297e40c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:38.707308 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087f997-c953-4c2a-a72d-2d0da3e1ee41 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:38.713043 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448acd58-2401-4e68-9f5b-8fe059cb69c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:38.713873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448acd58-2401-4e68-9f5b-8fe059cb69c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:38.714237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448acd58-2401-4e68-9f5b-8fe059cb69c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:38.714720 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448acd58-2401-4e68-9f5b-8fe059cb69c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:38.715289 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448acd58-2401-4e68-9f5b-8fe059cb69c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:38.717349 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448acd58-2401-4e68-9f5b-8fe059cb69c0 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:38.728773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1293/2583] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:38:38.733102 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448acd58-2401-4e68-9f5b-8fe059cb69c0 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:38.739771 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ae3a1-73c6-4131-af60-33d02f8b1c60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:38.742475 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ae3a1-73c6-4131-af60-33d02f8b1c60 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:38.742769 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ae3a1-73c6-4131-af60-33d02f8b1c60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:38.743133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ae3a1-73c6-4131-af60-33d02f8b1c60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:38.744343 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448acd58-2401-4e68-9f5b-8fe059cb69c0 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:38.750083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0735ad-0b14-45aa-89f1-096c4d6780f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:38.751043 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0735ad-0b14-45aa-89f1-096c4d6780f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:38.751551 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0735ad-0b14-45aa-89f1-096c4d6780f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:38.752017 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0735ad-0b14-45aa-89f1-096c4d6780f2 admin admin] PATH_INFO: `/v3/projects/ce366276b9044814aa1d82a6dff5bf49/users/43bd9c7694e642f88963d9aef84a1f66/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:38.754164 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0735ad-0b14-45aa-89f1-096c4d6780f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce366276b9044814aa1d82a6dff5bf49, user_id=43bd9c7694e642f88963d9aef84a1f66, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:38.757362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0735ad-0b14-45aa-89f1-096c4d6780f2 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:38.766240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ae3a1-73c6-4131-af60-33d02f8b1c60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:38.771209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ae3a1-73c6-4131-af60-33d02f8b1c60 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:38.775186 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1294/2584] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:38 2022] PUT /identity/v3/projects/ce366276b9044814aa1d82a6dff5bf49/users/43bd9c7694e642f88963d9aef84a1f66/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:38.786652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1291/2585] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:38.787004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecd34f-bf97-4e44-942d-f4b943ec16d7 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:38.787291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecd34f-bf97-4e44-942d-f4b943ec16d7 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:38.787558 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecd34f-bf97-4e44-942d-f4b943ec16d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:38.837507 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-83ecd34f-bf97-4e44-942d-f4b943ec16d7 None None] MFA Rules not processed for user `43bd9c7694e642f88963d9aef84a1f66`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:38.839736 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-83ecd34f-bf97-4e44-942d-f4b943ec16d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:38.989946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1295/2586] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:38.998296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ae3a1-73c6-4131-af60-33d02f8b1c60 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:38.999726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c31ae3a1-73c6-4131-af60-33d02f8b1c60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:39.098141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ae3a1-73c6-4131-af60-33d02f8b1c60 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:39.167403 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:39.181581 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f230dd7-6dda-4eab-a90a-6d8b059eb662 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:39.182483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f230dd7-6dda-4eab-a90a-6d8b059eb662 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:39.182941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f230dd7-6dda-4eab-a90a-6d8b059eb662 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:39.183365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f230dd7-6dda-4eab-a90a-6d8b059eb662 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:39.186251 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6f230dd7-6dda-4eab-a90a-6d8b059eb662 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:39.280227 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecd34f-bf97-4e44-942d-f4b943ec16d7 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:39.281767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-83ecd34f-bf97-4e44-942d-f4b943ec16d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:39.344016 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f230dd7-6dda-4eab-a90a-6d8b059eb662 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:39.345757 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f230dd7-6dda-4eab-a90a-6d8b059eb662 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:39.363787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1292/2587] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:39.383340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:39.501254 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecd34f-bf97-4e44-942d-f4b943ec16d7 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:39.525454 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:39.528656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:39.529105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:39.529584 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] PATH_INFO: `/v3/projects/b3d501d666b9446f97b6a6186bcfba8b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:39.532629 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3d501d666b9446f97b6a6186bcfba8b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:39.540371 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:39.670852 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3d501d666b9446f97b6a6186bcfba8b', 'request_id': 'req-47270466-7575-4aa6-baae-66f30d990144'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:39.677660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3d501d666b9446f97b6a6186bcfba8b', 'request_id': 'req-47270466-7575-4aa6-baae-66f30d990144'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:39.715247 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3d501d666b9446f97b6a6186bcfba8b', 'request_id': 'req-47270466-7575-4aa6-baae-66f30d990144'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:39.737988 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] The token cache is being invalidate because project b3d501d666b9446f97b6a6186bcfba8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:39.738957 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:39.739376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:39.746587 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1296/2588] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:39 2022] DELETE /identity/v3/projects/b3d501d666b9446f97b6a6186bcfba8b => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:39.749113 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f230dd7-6dda-4eab-a90a-6d8b059eb662 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:39.771345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f230dd7-6dda-4eab-a90a-6d8b059eb662 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:39.902803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5af28-980c-4c70-b4fb-f64dbfb2c592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:39.903394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5af28-980c-4c70-b4fb-f64dbfb2c592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:39.903829 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5af28-980c-4c70-b4fb-f64dbfb2c592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:39.903951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5af28-980c-4c70-b4fb-f64dbfb2c592 admin admin] PATH_INFO: `/v3/users/6eb2cad831584fa08d8b928737e1bb3e` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:39.910565 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5af28-980c-4c70-b4fb-f64dbfb2c592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eb2cad831584fa08d8b928737e1bb3e)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:39.910565 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5af28-980c-4c70-b4fb-f64dbfb2c592 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:40.002938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eda5af28-980c-4c70-b4fb-f64dbfb2c592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eb2cad831584fa08d8b928737e1bb3e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:40.003103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eda5af28-980c-4c70-b4fb-f64dbfb2c592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eb2cad831584fa08d8b928737e1bb3e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:40.003725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eda5af28-980c-4c70-b4fb-f64dbfb2c592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eb2cad831584fa08d8b928737e1bb3e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:40.012335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eda5af28-980c-4c70-b4fb-f64dbfb2c592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eb2cad831584fa08d8b928737e1bb3e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:40.028979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-eda5af28-980c-4c70-b4fb-f64dbfb2c592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eb2cad831584fa08d8b928737e1bb3e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:40.055654 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1293/2589] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:39 2022] DELETE /identity/v3/users/6eb2cad831584fa08d8b928737e1bb3e => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:40.361837 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:40.363996 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:40.386166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47270466-7575-4aa6-baae-66f30d990144 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:40.469261 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddcc71-ca05-40fd-893e-0a29e5bd996b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:40.470307 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddcc71-ca05-40fd-893e-0a29e5bd996b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:40.471615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddcc71-ca05-40fd-893e-0a29e5bd996b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:40.471964 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddcc71-ca05-40fd-893e-0a29e5bd996b admin admin] PATH_INFO: `/v3/projects/88a8b93c88af490084f6153c5c430419` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:40.473434 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddcc71-ca05-40fd-893e-0a29e5bd996b admin admin] RBAC: Authorizing `identity:delete_project(project_id=88a8b93c88af490084f6153c5c430419)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:40.475483 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddcc71-ca05-40fd-893e-0a29e5bd996b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:40.535771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-92ddcc71-ca05-40fd-893e-0a29e5bd996b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88a8b93c88af490084f6153c5c430419', 'request_id': 'req-92ddcc71-ca05-40fd-893e-0a29e5bd996b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:40.543323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-92ddcc71-ca05-40fd-893e-0a29e5bd996b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88a8b93c88af490084f6153c5c430419', 'request_id': 'req-92ddcc71-ca05-40fd-893e-0a29e5bd996b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:40.547571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:40.552092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-92ddcc71-ca05-40fd-893e-0a29e5bd996b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88a8b93c88af490084f6153c5c430419', 'request_id': 'req-92ddcc71-ca05-40fd-893e-0a29e5bd996b'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:40.567654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-92ddcc71-ca05-40fd-893e-0a29e5bd996b admin admin] The token cache is being invalidate because project 88a8b93c88af490084f6153c5c430419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:40.568180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-92ddcc71-ca05-40fd-893e-0a29e5bd996b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:40.569286 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-92ddcc71-ca05-40fd-893e-0a29e5bd996b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:40.571206 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1297/2590] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:40 2022] DELETE /identity/v3/projects/88a8b93c88af490084f6153c5c430419 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:40.595784 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d122858-342b-403c-ba89-312025a25e41 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:40.596168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d122858-342b-403c-ba89-312025a25e41 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:40.602944 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d122858-342b-403c-ba89-312025a25e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:40.617338 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0d122858-342b-403c-ba89-312025a25e41 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:40.618674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0d122858-342b-403c-ba89-312025a25e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:40.628332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:40.688928 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1294/2591] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:40.694771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1931f-11ea-4f86-944a-42ca82ab9ed7 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:40.695049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1931f-11ea-4f86-944a-42ca82ab9ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:40.695315 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1931f-11ea-4f86-944a-42ca82ab9ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:40.715395 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d8d1931f-11ea-4f86-944a-42ca82ab9ed7 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:40.716603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d8d1931f-11ea-4f86-944a-42ca82ab9ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:40.724442 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1298/2592] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:40.728879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d122858-342b-403c-ba89-312025a25e41 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:40.744027 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d122858-342b-403c-ba89-312025a25e41 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:40.750549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88f4df-1dcc-44c1-a0b2-7701d8a113a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:40.751207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88f4df-1dcc-44c1-a0b2-7701d8a113a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:40.751434 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88f4df-1dcc-44c1-a0b2-7701d8a113a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:40.751699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88f4df-1dcc-44c1-a0b2-7701d8a113a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:40.752494 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88f4df-1dcc-44c1-a0b2-7701d8a113a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:38:40.752794 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88f4df-1dcc-44c1-a0b2-7701d8a113a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:40.754534 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88f4df-1dcc-44c1-a0b2-7701d8a113a3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:40.772706 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1295/2593] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:38:40.777660 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1931f-11ea-4f86-944a-42ca82ab9ed7 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:40.789271 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1931f-11ea-4f86-944a-42ca82ab9ed7 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:40.804121 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643a252-df78-4520-bed1-d6dabb4ac5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:40.805103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643a252-df78-4520-bed1-d6dabb4ac5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:40.805321 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643a252-df78-4520-bed1-d6dabb4ac5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:40.805623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643a252-df78-4520-bed1-d6dabb4ac5d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:40.806258 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643a252-df78-4520-bed1-d6dabb4ac5d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:40.808338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643a252-df78-4520-bed1-d6dabb4ac5d9 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:40.873184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88f4df-1dcc-44c1-a0b2-7701d8a113a3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:40.874196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1299/2594] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:40 2022] POST /identity/v3/projects => generated 393 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:40.884150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88f4df-1dcc-44c1-a0b2-7701d8a113a3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:40.985229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e795c99-20cb-407d-ac59-6a4a87a214cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:40.986133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e795c99-20cb-407d-ac59-6a4a87a214cd admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:40.995414 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e795c99-20cb-407d-ac59-6a4a87a214cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:40.995939 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e795c99-20cb-407d-ac59-6a4a87a214cd admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:40.996807 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e795c99-20cb-407d-ac59-6a4a87a214cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:40.999138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e795c99-20cb-407d-ac59-6a4a87a214cd admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:41.041969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1296/2595] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:40 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:41.046488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643a252-df78-4520-bed1-d6dabb4ac5d9 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:41.060085 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643a252-df78-4520-bed1-d6dabb4ac5d9 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:41.067459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e31a5b-5e18-4278-a7e8-7b15f84803bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:41.068209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e31a5b-5e18-4278-a7e8-7b15f84803bc admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:41.068555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e31a5b-5e18-4278-a7e8-7b15f84803bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:41.068983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e31a5b-5e18-4278-a7e8-7b15f84803bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:41.069868 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e31a5b-5e18-4278-a7e8-7b15f84803bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:41.071626 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e31a5b-5e18-4278-a7e8-7b15f84803bc admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:41.109301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1300/2596] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:41.110322 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e795c99-20cb-407d-ac59-6a4a87a214cd admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:41.129089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e795c99-20cb-407d-ac59-6a4a87a214cd admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:41.153686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04a96c-89e3-4ef3-99fd-6008ed1630db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:41.153686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04a96c-89e3-4ef3-99fd-6008ed1630db admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:41.153686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04a96c-89e3-4ef3-99fd-6008ed1630db admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:41.153686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04a96c-89e3-4ef3-99fd-6008ed1630db admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:41.154235 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04a96c-89e3-4ef3-99fd-6008ed1630db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:41.154235 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04a96c-89e3-4ef3-99fd-6008ed1630db admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:41.159300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1297/2597] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:41.167027 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e31a5b-5e18-4278-a7e8-7b15f84803bc admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:41.190208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e31a5b-5e18-4278-a7e8-7b15f84803bc admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:41.196304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b5b01-ab00-47d6-9f27-4f664300a74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:41.197233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b5b01-ab00-47d6-9f27-4f664300a74a admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:41.197730 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b5b01-ab00-47d6-9f27-4f664300a74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:41.198219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b5b01-ab00-47d6-9f27-4f664300a74a admin admin] PATH_INFO: `/v3/projects/038191fb6f1f4f8b9e59b19cef32661e/users/fdd05b9b6dd04e73af0358af84d8d615/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:41.218450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b5b01-ab00-47d6-9f27-4f664300a74a admin admin] RBAC: Authorizing `identity:create_grant(project_id=038191fb6f1f4f8b9e59b19cef32661e, user_id=fdd05b9b6dd04e73af0358af84d8d615, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:41.230693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b5b01-ab00-47d6-9f27-4f664300a74a admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:41.257926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1301/2598] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:41 2022] PUT /identity/v3/projects/038191fb6f1f4f8b9e59b19cef32661e/users/fdd05b9b6dd04e73af0358af84d8d615/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:41.260191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba61b5-9f4c-487a-a85d-729fe6f6fccb None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:41.260728 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba61b5-9f4c-487a-a85d-729fe6f6fccb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:41.261198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba61b5-9f4c-487a-a85d-729fe6f6fccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:41.306317 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f3ba61b5-9f4c-487a-a85d-729fe6f6fccb None None] MFA Rules not processed for user `fdd05b9b6dd04e73af0358af84d8d615`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:41.307699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f3ba61b5-9f4c-487a-a85d-729fe6f6fccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:41.349705 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b5b01-ab00-47d6-9f27-4f664300a74a admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:41.351911 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-8e5b5b01-ab00-47d6-9f27-4f664300a74a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:41.419647 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1298/2599] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:41.420679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba61b5-9f4c-487a-a85d-729fe6f6fccb None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:41.436954 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:41.449234 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b5b01-ab00-47d6-9f27-4f664300a74a admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:41.465757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc0087-5251-48d1-b66b-8a3a0a1ca405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:41.465757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc0087-5251-48d1-b66b-8a3a0a1ca405 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:41.465757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc0087-5251-48d1-b66b-8a3a0a1ca405 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:41.465757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc0087-5251-48d1-b66b-8a3a0a1ca405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:41.476024 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-00fc0087-5251-48d1-b66b-8a3a0a1ca405 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:41.524613 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba61b5-9f4c-487a-a85d-729fe6f6fccb None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:41.534032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32bd18-57fc-421f-8c53-daebfadccabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:41.534987 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32bd18-57fc-421f-8c53-daebfadccabe admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:41.535334 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32bd18-57fc-421f-8c53-daebfadccabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:41.535810 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32bd18-57fc-421f-8c53-daebfadccabe admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:41.536633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32bd18-57fc-421f-8c53-daebfadccabe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:41.539119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32bd18-57fc-421f-8c53-daebfadccabe admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:41.574361 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1299/2600] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:41 2022] POST /identity/v3/projects => generated 393 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:41.578866 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32bd18-57fc-421f-8c53-daebfadccabe admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:41.580924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc0087-5251-48d1-b66b-8a3a0a1ca405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:41.582540 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc0087-5251-48d1-b66b-8a3a0a1ca405 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:41.592427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1302/2601] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:38:41.656163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32bd18-57fc-421f-8c53-daebfadccabe admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:41.661445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c78ab-166c-4412-b017-0d3d274b178a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:41.662409 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c78ab-166c-4412-b017-0d3d274b178a admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:41.662778 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c78ab-166c-4412-b017-0d3d274b178a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:41.663110 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c78ab-166c-4412-b017-0d3d274b178a admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:41.664164 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c78ab-166c-4412-b017-0d3d274b178a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:41.666193 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c78ab-166c-4412-b017-0d3d274b178a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:41.694540 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1300/2602] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:41 2022] POST /identity/v3/users => generated 303 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:41.698571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc0087-5251-48d1-b66b-8a3a0a1ca405 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:41.709842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc0087-5251-48d1-b66b-8a3a0a1ca405 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:41.716933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26c9ae-49f4-403a-aeda-87774d3020cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:41.717668 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26c9ae-49f4-403a-aeda-87774d3020cc admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:41.717943 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26c9ae-49f4-403a-aeda-87774d3020cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:41.718208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26c9ae-49f4-403a-aeda-87774d3020cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:41.718968 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26c9ae-49f4-403a-aeda-87774d3020cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:41.720697 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26c9ae-49f4-403a-aeda-87774d3020cc admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:41.731051 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1303/2603] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:38:41.735421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c78ab-166c-4412-b017-0d3d274b178a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:41.746199 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c78ab-166c-4412-b017-0d3d274b178a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:41.751109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decb924-e4a9-49a9-b9f7-992f37902a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:41.751766 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decb924-e4a9-49a9-b9f7-992f37902a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:41.752045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decb924-e4a9-49a9-b9f7-992f37902a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:41.752320 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decb924-e4a9-49a9-b9f7-992f37902a06 admin admin] PATH_INFO: `/v3/projects/80df364a1c1b4344980da8cd730fa112/users/357c59ea50af4b26bc89028104c1b6a0/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:41.771487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decb924-e4a9-49a9-b9f7-992f37902a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80df364a1c1b4344980da8cd730fa112, user_id=357c59ea50af4b26bc89028104c1b6a0, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:41.774447 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decb924-e4a9-49a9-b9f7-992f37902a06 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:41.792476 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1301/2604] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:41 2022] PUT /identity/v3/projects/80df364a1c1b4344980da8cd730fa112/users/357c59ea50af4b26bc89028104c1b6a0/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:41.796647 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26c9ae-49f4-403a-aeda-87774d3020cc admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:41.807449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26c9ae-49f4-403a-aeda-87774d3020cc admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:41.834845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:41.880694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcaae7-5dab-44dd-a7fc-be614da3c01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:41.881309 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcaae7-5dab-44dd-a7fc-be614da3c01c admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:41.881612 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcaae7-5dab-44dd-a7fc-be614da3c01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:41.881855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcaae7-5dab-44dd-a7fc-be614da3c01c admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:41.882488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcaae7-5dab-44dd-a7fc-be614da3c01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:41.884141 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcaae7-5dab-44dd-a7fc-be614da3c01c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:41.894472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1304/2605] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:41.898875 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decb924-e4a9-49a9-b9f7-992f37902a06 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:41.909355 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decb924-e4a9-49a9-b9f7-992f37902a06 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:41.914355 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131cc86-c31b-4043-828a-71e4d85ea379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:41.915036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131cc86-c31b-4043-828a-71e4d85ea379 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:41.915310 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131cc86-c31b-4043-828a-71e4d85ea379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:41.915578 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131cc86-c31b-4043-828a-71e4d85ea379 admin admin] PATH_INFO: `/v3/projects/80df364a1c1b4344980da8cd730fa112/users/357c59ea50af4b26bc89028104c1b6a0/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:41.917482 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131cc86-c31b-4043-828a-71e4d85ea379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80df364a1c1b4344980da8cd730fa112, user_id=357c59ea50af4b26bc89028104c1b6a0, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:41.920239 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131cc86-c31b-4043-828a-71e4d85ea379 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:41.931086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1302/2606] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:41 2022] PUT /identity/v3/projects/80df364a1c1b4344980da8cd730fa112/users/357c59ea50af4b26bc89028104c1b6a0/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:41.941735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39d5e0-3b0c-46cd-84c6-31da81764b30 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:41.942015 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39d5e0-3b0c-46cd-84c6-31da81764b30 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:41.942321 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39d5e0-3b0c-46cd-84c6-31da81764b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:41.976506 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-7d39d5e0-3b0c-46cd-84c6-31da81764b30 None None] MFA Rules not processed for user `357c59ea50af4b26bc89028104c1b6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:41.977701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-7d39d5e0-3b0c-46cd-84c6-31da81764b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:42.093665 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1305/2607] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:42.107542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131cc86-c31b-4043-828a-71e4d85ea379 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:42.109240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c131cc86-c31b-4043-828a-71e4d85ea379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:42.193237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:42.206722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131cc86-c31b-4043-828a-71e4d85ea379 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:42.212824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec74a7-f31f-41ad-a026-4302478ecbb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:42.214152 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec74a7-f31f-41ad-a026-4302478ecbb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:42.214428 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec74a7-f31f-41ad-a026-4302478ecbb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:42.214717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec74a7-f31f-41ad-a026-4302478ecbb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:42.246721 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec74a7-f31f-41ad-a026-4302478ecbb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:42.248520 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec74a7-f31f-41ad-a026-4302478ecbb4 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:42.274961 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1303/2608] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:42 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:42.327195 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39d5e0-3b0c-46cd-84c6-31da81764b30 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:42.341109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39d5e0-3b0c-46cd-84c6-31da81764b30 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:42.349758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ab440-0e97-42af-a894-0d5a22cd1ee4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:42.349758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ab440-0e97-42af-a894-0d5a22cd1ee4 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:42.349758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ab440-0e97-42af-a894-0d5a22cd1ee4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:42.349758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ab440-0e97-42af-a894-0d5a22cd1ee4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:42.355084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ea7ab440-0e97-42af-a894-0d5a22cd1ee4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:42.357475 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec74a7-f31f-41ad-a026-4302478ecbb4 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:42.367410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec74a7-f31f-41ad-a026-4302478ecbb4 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:42.371733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0acf2-1a40-4f80-b24e-ccafbbecaa59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:42.372351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0acf2-1a40-4f80-b24e-ccafbbecaa59 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:42.372625 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0acf2-1a40-4f80-b24e-ccafbbecaa59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:42.372893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0acf2-1a40-4f80-b24e-ccafbbecaa59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:42.459753 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0acf2-1a40-4f80-b24e-ccafbbecaa59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:42.462635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0acf2-1a40-4f80-b24e-ccafbbecaa59 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:42.463930 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ab440-0e97-42af-a894-0d5a22cd1ee4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:42.466328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ab440-0e97-42af-a894-0d5a22cd1ee4 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:42.474343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1304/2609] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:42 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:38:42.482691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1306/2610] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:45.906833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0acf2-1a40-4f80-b24e-ccafbbecaa59 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:45.978343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0acf2-1a40-4f80-b24e-ccafbbecaa59 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:45.983929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26202c2a-dc22-4298-bea4-f4306de6afd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:45.984569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26202c2a-dc22-4298-bea4-f4306de6afd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:45.984847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26202c2a-dc22-4298-bea4-f4306de6afd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:45.985167 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26202c2a-dc22-4298-bea4-f4306de6afd8 admin admin] PATH_INFO: `/v3/users/fdd05b9b6dd04e73af0358af84d8d615` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:45.986546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26202c2a-dc22-4298-bea4-f4306de6afd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdd05b9b6dd04e73af0358af84d8d615)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:45.988478 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26202c2a-dc22-4298-bea4-f4306de6afd8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:46.054964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-26202c2a-dc22-4298-bea4-f4306de6afd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdd05b9b6dd04e73af0358af84d8d615'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.055630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-26202c2a-dc22-4298-bea4-f4306de6afd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdd05b9b6dd04e73af0358af84d8d615'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.056229 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-26202c2a-dc22-4298-bea4-f4306de6afd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdd05b9b6dd04e73af0358af84d8d615'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.064847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-26202c2a-dc22-4298-bea4-f4306de6afd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdd05b9b6dd04e73af0358af84d8d615'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.101670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-26202c2a-dc22-4298-bea4-f4306de6afd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdd05b9b6dd04e73af0358af84d8d615'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.122746 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1305/2611] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:45 2022] DELETE /identity/v3/users/fdd05b9b6dd04e73af0358af84d8d615 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:46.125284 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a092a5-3172-43a8-97ae-e26a03fa9caf None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:46.125750 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a092a5-3172-43a8-97ae-e26a03fa9caf None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:46.125867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a092a5-3172-43a8-97ae-e26a03fa9caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:46.148174 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-f4a092a5-3172-43a8-97ae-e26a03fa9caf None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:46.149501 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f4a092a5-3172-43a8-97ae-e26a03fa9caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:46.194457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:46.242294 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1307/2612] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:46.256210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26202c2a-dc22-4298-bea4-f4306de6afd8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:46.257617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-26202c2a-dc22-4298-bea4-f4306de6afd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:46.272936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:46.346697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26202c2a-dc22-4298-bea4-f4306de6afd8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:46.353458 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b98c8-2659-4f29-a714-05ac221470ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:46.354189 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b98c8-2659-4f29-a714-05ac221470ef service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:46.354632 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b98c8-2659-4f29-a714-05ac221470ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:46.354907 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b98c8-2659-4f29-a714-05ac221470ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:46.369017 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b98c8-2659-4f29-a714-05ac221470ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:46.370533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b98c8-2659-4f29-a714-05ac221470ef service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:46.380815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1306/2613] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:46.518126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a092a5-3172-43a8-97ae-e26a03fa9caf None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:46.520308 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f4a092a5-3172-43a8-97ae-e26a03fa9caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:46.536804 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a092a5-3172-43a8-97ae-e26a03fa9caf None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:46.591461 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9ed70-8c02-4301-92b9-45c4585924b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:46.592370 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9ed70-8c02-4301-92b9-45c4585924b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:46.592739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9ed70-8c02-4301-92b9-45c4585924b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:46.593157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9ed70-8c02-4301-92b9-45c4585924b7 admin admin] PATH_INFO: `/v3/users/f0a6c8454bdb4f54a1a5dc38381e7e28` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:46.597884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9ed70-8c02-4301-92b9-45c4585924b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0a6c8454bdb4f54a1a5dc38381e7e28)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:46.600242 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9ed70-8c02-4301-92b9-45c4585924b7 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:46.670610 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1fd9ed70-8c02-4301-92b9-45c4585924b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0a6c8454bdb4f54a1a5dc38381e7e28'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.671931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1fd9ed70-8c02-4301-92b9-45c4585924b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0a6c8454bdb4f54a1a5dc38381e7e28'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.672595 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1fd9ed70-8c02-4301-92b9-45c4585924b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0a6c8454bdb4f54a1a5dc38381e7e28'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.682368 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1fd9ed70-8c02-4301-92b9-45c4585924b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0a6c8454bdb4f54a1a5dc38381e7e28'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.698325 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1fd9ed70-8c02-4301-92b9-45c4585924b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0a6c8454bdb4f54a1a5dc38381e7e28'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.721388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1308/2614] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:46 2022] DELETE /identity/v3/users/f0a6c8454bdb4f54a1a5dc38381e7e28 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:46.726920 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27623b-6e75-4bae-ba23-fe2458f5cd98 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:46.727702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27623b-6e75-4bae-ba23-fe2458f5cd98 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:46.728461 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27623b-6e75-4bae-ba23-fe2458f5cd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:46.749784 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-fc27623b-6e75-4bae-ba23-fe2458f5cd98 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:46.751150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-fc27623b-6e75-4bae-ba23-fe2458f5cd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:46.790651 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9ed70-8c02-4301-92b9-45c4585924b7 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:46.792619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1fd9ed70-8c02-4301-92b9-45c4585924b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:46.833668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1307/2615] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:46.844758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27623b-6e75-4bae-ba23-fe2458f5cd98 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:46.846165 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-fc27623b-6e75-4bae-ba23-fe2458f5cd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:46.890948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9ed70-8c02-4301-92b9-45c4585924b7 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:46.897291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:46.897291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:46.897291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:46.897634 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] PATH_INFO: `/v3/projects/038191fb6f1f4f8b9e59b19cef32661e` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:46.898818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=038191fb6f1f4f8b9e59b19cef32661e)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:46.901277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:46.924313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27623b-6e75-4bae-ba23-fe2458f5cd98 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:46.932585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a93b7-1559-48c7-ac02-0bbd6292a77b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:46.935103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a93b7-1559-48c7-ac02-0bbd6292a77b service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:46.935676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a93b7-1559-48c7-ac02-0bbd6292a77b service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:46.936152 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a93b7-1559-48c7-ac02-0bbd6292a77b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:46.949020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a93b7-1559-48c7-ac02-0bbd6292a77b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:46.950480 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a93b7-1559-48c7-ac02-0bbd6292a77b service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:46.957401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '038191fb6f1f4f8b9e59b19cef32661e', 'request_id': 'req-3659cac5-2982-4e36-a37e-a2482c6f5df1'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.960742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1308/2616] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:38:46.965982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '038191fb6f1f4f8b9e59b19cef32661e', 'request_id': 'req-3659cac5-2982-4e36-a37e-a2482c6f5df1'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.969926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:46.976088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '038191fb6f1f4f8b9e59b19cef32661e', 'request_id': 'req-3659cac5-2982-4e36-a37e-a2482c6f5df1'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.989361 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] The token cache is being invalidate because project 038191fb6f1f4f8b9e59b19cef32661e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:46.989677 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.990685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:46.992403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1309/2617] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:46 2022] DELETE /identity/v3/projects/038191fb6f1f4f8b9e59b19cef32661e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:46.996419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a93b7-1559-48c7-ac02-0bbd6292a77b service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:47.008929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a93b7-1559-48c7-ac02-0bbd6292a77b service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:47.072674 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:47.082996 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ab98c-f598-4270-8fbd-925b6c748486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:47.083874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ab98c-f598-4270-8fbd-925b6c748486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:47.084442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ab98c-f598-4270-8fbd-925b6c748486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:47.085039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ab98c-f598-4270-8fbd-925b6c748486 admin admin] PATH_INFO: `/v3/users/357c59ea50af4b26bc89028104c1b6a0` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:47.086604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ab98c-f598-4270-8fbd-925b6c748486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=357c59ea50af4b26bc89028104c1b6a0)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:47.088735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ab98c-f598-4270-8fbd-925b6c748486 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:47.159986 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bc1ab98c-f598-4270-8fbd-925b6c748486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '357c59ea50af4b26bc89028104c1b6a0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.160709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bc1ab98c-f598-4270-8fbd-925b6c748486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '357c59ea50af4b26bc89028104c1b6a0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.161538 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bc1ab98c-f598-4270-8fbd-925b6c748486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '357c59ea50af4b26bc89028104c1b6a0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.180032 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bc1ab98c-f598-4270-8fbd-925b6c748486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '357c59ea50af4b26bc89028104c1b6a0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.203134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bc1ab98c-f598-4270-8fbd-925b6c748486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '357c59ea50af4b26bc89028104c1b6a0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.218558 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1309/2618] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:46 2022] DELETE /identity/v3/users/357c59ea50af4b26bc89028104c1b6a0 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:47.284322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:47.285900 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:47.303781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659cac5-2982-4e36-a37e-a2482c6f5df1 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:47.387220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:47.387991 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:47.388274 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:47.388544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] PATH_INFO: `/v3/projects/531efc4f316f453fb01618e7a472f4b4` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:47.389806 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] RBAC: Authorizing `identity:delete_project(project_id=531efc4f316f453fb01618e7a472f4b4)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:47.391884 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:47.440335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc6f43-e28c-4b63-b1dc-2f0d9ecbf5bb None None] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:47.440617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc6f43-e28c-4b63-b1dc-2f0d9ecbf5bb None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:47.440895 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc6f43-e28c-4b63-b1dc-2f0d9ecbf5bb None None] PATH_INFO: `/` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:47.442253 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1310/2619] 10.222.0.23 () {54 vars in 921 bytes} [Fri Sep 9 16:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:38:47.458036 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '531efc4f316f453fb01618e7a472f4b4', 'request_id': 'req-f65dee0c-9c4d-4a3e-98db-d12702ac895e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.463579 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b356b-1438-48a9-bdf9-65e312d2a297 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:47.464047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b356b-1438-48a9-bdf9-65e312d2a297 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:47.464945 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b356b-1438-48a9-bdf9-65e312d2a297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:47.465098 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '531efc4f316f453fb01618e7a472f4b4', 'request_id': 'req-f65dee0c-9c4d-4a3e-98db-d12702ac895e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.481364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '531efc4f316f453fb01618e7a472f4b4', 'request_id': 'req-f65dee0c-9c4d-4a3e-98db-d12702ac895e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.486660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-dd2b356b-1438-48a9-bdf9-65e312d2a297 None None] MFA Rules not processed for user `152a41ad598b4b01b1881d928c53c848`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:47.487983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dd2b356b-1438-48a9-bdf9-65e312d2a297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:47.493510 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] The token cache is being invalidate because project 531efc4f316f453fb01618e7a472f4b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:47.493794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.494407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.496202 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1310/2620] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:47 2022] DELETE /identity/v3/projects/531efc4f316f453fb01618e7a472f4b4 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:47.502620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:47.504378 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:47.577335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1311/2621] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Sep 9 16:38:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:47.599926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dee0c-9c4d-4a3e-98db-d12702ac895e admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:47.604600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b356b-1438-48a9-bdf9-65e312d2a297 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:47.606052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dd2b356b-1438-48a9-bdf9-65e312d2a297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:47.621280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72872251-f8b5-4f72-b645-d29a14bb8735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:47.623610 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72872251-f8b5-4f72-b645-d29a14bb8735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:47.623771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72872251-f8b5-4f72-b645-d29a14bb8735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:47.623994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72872251-f8b5-4f72-b645-d29a14bb8735 admin admin] PATH_INFO: `/v3/users/b7a997640f994c40a16f5a7cfd71eb07` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:47.626964 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72872251-f8b5-4f72-b645-d29a14bb8735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a997640f994c40a16f5a7cfd71eb07)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:47.629861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72872251-f8b5-4f72-b645-d29a14bb8735 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:47.666215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:47.716425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b356b-1438-48a9-bdf9-65e312d2a297 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:47.726262 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6529cf-4afb-4604-8dd0-d8f80339ff0e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a2fb70c9ae4f4b81ac4de513b868cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:47.726927 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6529cf-4afb-4604-8dd0-d8f80339ff0e service glance] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:47.727208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6529cf-4afb-4604-8dd0-d8f80339ff0e service glance] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:47.727479 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6529cf-4afb-4604-8dd0-d8f80339ff0e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:47.728413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-72872251-f8b5-4f72-b645-d29a14bb8735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a997640f994c40a16f5a7cfd71eb07'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.729403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-72872251-f8b5-4f72-b645-d29a14bb8735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a997640f994c40a16f5a7cfd71eb07'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.730870 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-72872251-f8b5-4f72-b645-d29a14bb8735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a997640f994c40a16f5a7cfd71eb07'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.744206 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-72872251-f8b5-4f72-b645-d29a14bb8735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a997640f994c40a16f5a7cfd71eb07'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.750097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6529cf-4afb-4604-8dd0-d8f80339ff0e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:47.751259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6529cf-4afb-4604-8dd0-d8f80339ff0e service glance] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:47.757807 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-72872251-f8b5-4f72-b645-d29a14bb8735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a997640f994c40a16f5a7cfd71eb07'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.764270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1312/2622] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:47.773121 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1311/2623] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:47 2022] DELETE /identity/v3/users/b7a997640f994c40a16f5a7cfd71eb07 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:47.779441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6529cf-4afb-4604-8dd0-d8f80339ff0e service glance] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:47.781833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4d6529cf-4afb-4604-8dd0-d8f80339ff0e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:47.796758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6529cf-4afb-4604-8dd0-d8f80339ff0e service glance] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:47.801828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:47.872801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219b47c-6df3-400d-8bc5-725cf2c514b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:47.873556 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219b47c-6df3-400d-8bc5-725cf2c514b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:47.873891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219b47c-6df3-400d-8bc5-725cf2c514b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:47.874323 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219b47c-6df3-400d-8bc5-725cf2c514b0 admin admin] PATH_INFO: `/v3/projects/80df364a1c1b4344980da8cd730fa112` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:47.875664 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219b47c-6df3-400d-8bc5-725cf2c514b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80df364a1c1b4344980da8cd730fa112)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:47.876517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c0a9c-2e24-49bc-b91a-79cad751a7d2 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:47.876842 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c0a9c-2e24-49bc-b91a-79cad751a7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:47.877153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c0a9c-2e24-49bc-b91a-79cad751a7d2 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:47.878227 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219b47c-6df3-400d-8bc5-725cf2c514b0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:47.878917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1312/2624] 10.222.0.23 () {54 vars in 921 bytes} [Fri Sep 9 16:38:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 16:38:47.884495 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43363d0d-2c29-43d7-8243-391816339e78 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:47.884779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43363d0d-2c29-43d7-8243-391816339e78 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:47.885050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43363d0d-2c29-43d7-8243-391816339e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:47.919109 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-43363d0d-2c29-43d7-8243-391816339e78 None None] MFA Rules not processed for user `aa65b6854dc54e289cc3e8e0b3746dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:47.920591 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-43363d0d-2c29-43d7-8243-391816339e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:47.955813 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-b219b47c-6df3-400d-8bc5-725cf2c514b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80df364a1c1b4344980da8cd730fa112', 'request_id': 'req-b219b47c-6df3-400d-8bc5-725cf2c514b0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.970106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-b219b47c-6df3-400d-8bc5-725cf2c514b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80df364a1c1b4344980da8cd730fa112', 'request_id': 'req-b219b47c-6df3-400d-8bc5-725cf2c514b0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.980413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1313/2625] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Sep 9 16:38:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:47.981154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-b219b47c-6df3-400d-8bc5-725cf2c514b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80df364a1c1b4344980da8cd730fa112', 'request_id': 'req-b219b47c-6df3-400d-8bc5-725cf2c514b0'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:47.994128 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43363d0d-2c29-43d7-8243-391816339e78 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:47.996676 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-43363d0d-2c29-43d7-8243-391816339e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:48.004345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-b219b47c-6df3-400d-8bc5-725cf2c514b0 admin admin] The token cache is being invalidate because project 80df364a1c1b4344980da8cd730fa112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:48.004816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-b219b47c-6df3-400d-8bc5-725cf2c514b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.005475 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-b219b47c-6df3-400d-8bc5-725cf2c514b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.007266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1313/2626] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:47 2022] DELETE /identity/v3/projects/80df364a1c1b4344980da8cd730fa112 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:48.012043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e250087-23bf-4ae7-a6c6-f21b208a2f15 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.012501 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e250087-23bf-4ae7-a6c6-f21b208a2f15 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.012703 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43363d0d-2c29-43d7-8243-391816339e78 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:48.012820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e250087-23bf-4ae7-a6c6-f21b208a2f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.035642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3e250087-23bf-4ae7-a6c6-f21b208a2f15 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:48.037601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3e250087-23bf-4ae7-a6c6-f21b208a2f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:48.063914 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d0d1a-1ff3-405e-ae3b-b196b8959991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:48.064535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d0d1a-1ff3-405e-ae3b-b196b8959991 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.064811 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d0d1a-1ff3-405e-ae3b-b196b8959991 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.065079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d0d1a-1ff3-405e-ae3b-b196b8959991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.066895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-4e6d0d1a-1ff3-405e-ae3b-b196b8959991 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:48.082749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d0d1a-1ff3-405e-ae3b-b196b8959991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:48.084774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d0d1a-1ff3-405e-ae3b-b196b8959991 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:48.094922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1314/2627] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:48.107455 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d0d1a-1ff3-405e-ae3b-b196b8959991 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:48.111348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-4e6d0d1a-1ff3-405e-ae3b-b196b8959991 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:48.119906 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1314/2628] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:48.128652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2bc3f-8528-486d-b2e1-884adf4fccc1 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.128934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2bc3f-8528-486d-b2e1-884adf4fccc1 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.129202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2bc3f-8528-486d-b2e1-884adf4fccc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.132579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d0d1a-1ff3-405e-ae3b-b196b8959991 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:48.137885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256abbe-e49f-4f9e-8271-a9d153685ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:48.138559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256abbe-e49f-4f9e-8271-a9d153685ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.139134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256abbe-e49f-4f9e-8271-a9d153685ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.139408 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256abbe-e49f-4f9e-8271-a9d153685ca4 admin admin] PATH_INFO: `/v3/projects/764af9d74f804d918a36c3f5ac851e4e` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.140878 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256abbe-e49f-4f9e-8271-a9d153685ca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=764af9d74f804d918a36c3f5ac851e4e)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:48.142825 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256abbe-e49f-4f9e-8271-a9d153685ca4 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:48.159650 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3ea2bc3f-8528-486d-b2e1-884adf4fccc1 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:48.160943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3ea2bc3f-8528-486d-b2e1-884adf4fccc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:48.171932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1315/2629] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:48.176297 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2bc3f-8528-486d-b2e1-884adf4fccc1 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:48.186171 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2bc3f-8528-486d-b2e1-884adf4fccc1 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:48.192487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:48.200092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-6256abbe-e49f-4f9e-8271-a9d153685ca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '764af9d74f804d918a36c3f5ac851e4e', 'request_id': 'req-6256abbe-e49f-4f9e-8271-a9d153685ca4'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.208784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-6256abbe-e49f-4f9e-8271-a9d153685ca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '764af9d74f804d918a36c3f5ac851e4e', 'request_id': 'req-6256abbe-e49f-4f9e-8271-a9d153685ca4'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.217149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-6256abbe-e49f-4f9e-8271-a9d153685ca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '764af9d74f804d918a36c3f5ac851e4e', 'request_id': 'req-6256abbe-e49f-4f9e-8271-a9d153685ca4'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.221561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:48.230697 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-6256abbe-e49f-4f9e-8271-a9d153685ca4 admin admin] The token cache is being invalidate because project 764af9d74f804d918a36c3f5ac851e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:48.230993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-6256abbe-e49f-4f9e-8271-a9d153685ca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.232200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-6256abbe-e49f-4f9e-8271-a9d153685ca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.233729 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1315/2630] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:48 2022] DELETE /identity/v3/projects/764af9d74f804d918a36c3f5ac851e4e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:48.237857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f2ce9-f10f-4cd6-a1da-2a4ce842f5e0 None None] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.237857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f2ce9-f10f-4cd6-a1da-2a4ce842f5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.237857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f2ce9-f10f-4cd6-a1da-2a4ce842f5e0 None None] PATH_INFO: `/` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.237857 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1316/2631] 10.222.0.23 () {54 vars in 921 bytes} [Fri Sep 9 16:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 16:38:48.238563 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f2ce9-f10f-4cd6-a1da-2a4ce842f5e0 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:48.241427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b90f2ce9-f10f-4cd6-a1da-2a4ce842f5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:48.254453 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bee8b6-8b1e-4b15-9b42-9920f2003f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:48.255526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bee8b6-8b1e-4b15-9b42-9920f2003f85 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.255818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bee8b6-8b1e-4b15-9b42-9920f2003f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.256087 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bee8b6-8b1e-4b15-9b42-9920f2003f85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.256737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bee8b6-8b1e-4b15-9b42-9920f2003f85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:38:48.257035 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bee8b6-8b1e-4b15-9b42-9920f2003f85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:48.258676 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bee8b6-8b1e-4b15-9b42-9920f2003f85 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:48.270733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1316/2632] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:38:48.272036 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76347b20-6f13-4870-871c-429f25d72bd4 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.272421 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76347b20-6f13-4870-871c-429f25d72bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.272695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76347b20-6f13-4870-871c-429f25d72bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.293060 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-76347b20-6f13-4870-871c-429f25d72bd4 None None] MFA Rules not processed for user `152a41ad598b4b01b1881d928c53c848`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:48.294253 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-76347b20-6f13-4870-871c-429f25d72bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:48.311653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f2ce9-f10f-4cd6-a1da-2a4ce842f5e0 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:48.317337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd282-202d-4c50-b642-59e37cf143dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:48.317977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd282-202d-4c50-b642-59e37cf143dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.318259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd282-202d-4c50-b642-59e37cf143dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.318556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd282-202d-4c50-b642-59e37cf143dc admin admin] PATH_INFO: `/v3/users/43bd9c7694e642f88963d9aef84a1f66` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.319789 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd282-202d-4c50-b642-59e37cf143dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=43bd9c7694e642f88963d9aef84a1f66)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:48.321729 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd282-202d-4c50-b642-59e37cf143dc admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:48.360839 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1317/2633] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Sep 9 16:38:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:48.361823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76347b20-6f13-4870-871c-429f25d72bd4 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:48.363311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-76347b20-6f13-4870-871c-429f25d72bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:48.369395 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e87cd282-202d-4c50-b642-59e37cf143dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43bd9c7694e642f88963d9aef84a1f66'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.370177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e87cd282-202d-4c50-b642-59e37cf143dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43bd9c7694e642f88963d9aef84a1f66'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.370864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e87cd282-202d-4c50-b642-59e37cf143dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43bd9c7694e642f88963d9aef84a1f66'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.377852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76347b20-6f13-4870-871c-429f25d72bd4 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:48.378685 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e87cd282-202d-4c50-b642-59e37cf143dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43bd9c7694e642f88963d9aef84a1f66'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.383112 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15217e7a-6006-4bb8-bcad-00574e716585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:48.383729 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15217e7a-6006-4bb8-bcad-00574e716585 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.384003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15217e7a-6006-4bb8-bcad-00574e716585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.384269 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15217e7a-6006-4bb8-bcad-00574e716585 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.384915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15217e7a-6006-4bb8-bcad-00574e716585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:48.386901 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15217e7a-6006-4bb8-bcad-00574e716585 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:48.391488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-e87cd282-202d-4c50-b642-59e37cf143dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43bd9c7694e642f88963d9aef84a1f66'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.408896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1317/2634] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:48 2022] DELETE /identity/v3/users/43bd9c7694e642f88963d9aef84a1f66 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:48.413416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd282-202d-4c50-b642-59e37cf143dc admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:48.414803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e87cd282-202d-4c50-b642-59e37cf143dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:48.417709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1318/2635] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:48 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:48.421546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15217e7a-6006-4bb8-bcad-00574e716585 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:48.423372 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-15217e7a-6006-4bb8-bcad-00574e716585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:48.472961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:48.478635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd282-202d-4c50-b642-59e37cf143dc admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:48.500326 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15217e7a-6006-4bb8-bcad-00574e716585 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:48.505907 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cc699-ab94-4f06-a29f-514a718af795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:48.506803 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cc699-ab94-4f06-a29f-514a718af795 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.507150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cc699-ab94-4f06-a29f-514a718af795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.507491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cc699-ab94-4f06-a29f-514a718af795 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.508542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cc699-ab94-4f06-a29f-514a718af795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:48.511349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cc699-ab94-4f06-a29f-514a718af795 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:48.535472 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1319/2636] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:48 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:48.540477 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cc699-ab94-4f06-a29f-514a718af795 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:48.540730 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2b531-93f2-41a0-9e1c-30b1cffb2b23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:48.541377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2b531-93f2-41a0-9e1c-30b1cffb2b23 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.541715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2b531-93f2-41a0-9e1c-30b1cffb2b23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.542043 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2b531-93f2-41a0-9e1c-30b1cffb2b23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.543910 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-90f2b531-93f2-41a0-9e1c-30b1cffb2b23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:48.554302 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cc699-ab94-4f06-a29f-514a718af795 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:48.560105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42768910-9f70-48a4-ac66-f15ee14f01e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:48.560428 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42768910-9f70-48a4-ac66-f15ee14f01e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.560800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42768910-9f70-48a4-ac66-f15ee14f01e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.561172 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42768910-9f70-48a4-ac66-f15ee14f01e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.561918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42768910-9f70-48a4-ac66-f15ee14f01e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:48.563654 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42768910-9f70-48a4-ac66-f15ee14f01e5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:48.576815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1320/2637] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:38:48.584258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42768910-9f70-48a4-ac66-f15ee14f01e5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:48.598466 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42768910-9f70-48a4-ac66-f15ee14f01e5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:48.604496 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f472b-2ece-45bd-affa-c09f78bfbde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:48.605464 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f472b-2ece-45bd-affa-c09f78bfbde6 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.605757 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f472b-2ece-45bd-affa-c09f78bfbde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.606008 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f472b-2ece-45bd-affa-c09f78bfbde6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.606682 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f472b-2ece-45bd-affa-c09f78bfbde6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:48.608314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f472b-2ece-45bd-affa-c09f78bfbde6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:48.619426 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1321/2638] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:48.623612 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f472b-2ece-45bd-affa-c09f78bfbde6 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:48.629419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2b531-93f2-41a0-9e1c-30b1cffb2b23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:48.629628 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:48.630899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2b531-93f2-41a0-9e1c-30b1cffb2b23 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:48.633911 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f472b-2ece-45bd-affa-c09f78bfbde6 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:48.640146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3296c54-8e35-4af7-899b-23c82ea0e0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:48.640827 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3296c54-8e35-4af7-899b-23c82ea0e0c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.641205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3296c54-8e35-4af7-899b-23c82ea0e0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.641490 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3296c54-8e35-4af7-899b-23c82ea0e0c8 admin admin] PATH_INFO: `/v3/projects/550daf0d9c1a4920aeb2fe44a9cfe10a/users/3fd8252e16014e7b9ccb9bfa7ab3bd6f/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.650635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1318/2639] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:48.658922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3296c54-8e35-4af7-899b-23c82ea0e0c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=550daf0d9c1a4920aeb2fe44a9cfe10a, user_id=3fd8252e16014e7b9ccb9bfa7ab3bd6f, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:48.660917 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3296c54-8e35-4af7-899b-23c82ea0e0c8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:48.691319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1322/2640] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:48 2022] PUT /identity/v3/projects/550daf0d9c1a4920aeb2fe44a9cfe10a/users/3fd8252e16014e7b9ccb9bfa7ab3bd6f/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:48.696633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81235daf-0c95-44c0-8617-d6a796d73e96 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.697018 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81235daf-0c95-44c0-8617-d6a796d73e96 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.697396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81235daf-0c95-44c0-8617-d6a796d73e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.758877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-81235daf-0c95-44c0-8617-d6a796d73e96 None None] MFA Rules not processed for user `3fd8252e16014e7b9ccb9bfa7ab3bd6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:48.760528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3296c54-8e35-4af7-899b-23c82ea0e0c8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:48.762278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f3296c54-8e35-4af7-899b-23c82ea0e0c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:48.769000 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-81235daf-0c95-44c0-8617-d6a796d73e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:48.865610 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3296c54-8e35-4af7-899b-23c82ea0e0c8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:48.871481 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:48.872126 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.872508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.872781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8 admin admin] PATH_INFO: `/v3/projects/ce366276b9044814aa1d82a6dff5bf49` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:48.874082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce366276b9044814aa1d82a6dff5bf49)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:48.877505 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:48.910626 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1319/2641] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:48.915534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81235daf-0c95-44c0-8617-d6a796d73e96 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:48.930088 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81235daf-0c95-44c0-8617-d6a796d73e96 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:48.940805 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce366276b9044814aa1d82a6dff5bf49', 'request_id': 'req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.948814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce366276b9044814aa1d82a6dff5bf49', 'request_id': 'req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.955358 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce366276b9044814aa1d82a6dff5bf49', 'request_id': 'req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.971747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8 admin admin] The token cache is being invalidate because project ce366276b9044814aa1d82a6dff5bf49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:48.972472 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.973741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-5dd243a4-6226-416c-b9c1-5ef22a8cf0b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:48.975905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1323/2642] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:48 2022] DELETE /identity/v3/projects/ce366276b9044814aa1d82a6dff5bf49 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:48.979703 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:48.980838 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b80ec9-6787-429f-bc41-2fc56fc2d34c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:48.982094 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b80ec9-6787-429f-bc41-2fc56fc2d34c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:48.982372 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b80ec9-6787-429f-bc41-2fc56fc2d34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.014743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-88b80ec9-6787-429f-bc41-2fc56fc2d34c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:49.014895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98c81a-f2cf-4b17-bb50-da3ddf927994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:49.015043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-88b80ec9-6787-429f-bc41-2fc56fc2d34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:49.015149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98c81a-f2cf-4b17-bb50-da3ddf927994 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.015149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98c81a-f2cf-4b17-bb50-da3ddf927994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.015149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98c81a-f2cf-4b17-bb50-da3ddf927994 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.015149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea98c81a-f2cf-4b17-bb50-da3ddf927994 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:49.016129 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea98c81a-f2cf-4b17-bb50-da3ddf927994 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:49.022758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1324/2643] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:49.031207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10548-634c-43fa-96f9-1cdd24635617 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.031494 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10548-634c-43fa-96f9-1cdd24635617 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.031762 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10548-634c-43fa-96f9-1cdd24635617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.050564 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1320/2644] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:48 2022] POST /identity/v3/projects => generated 381 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:49.054466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98c81a-f2cf-4b17-bb50-da3ddf927994 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:49.054610 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-1fd10548-634c-43fa-96f9-1cdd24635617 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:49.055929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1fd10548-634c-43fa-96f9-1cdd24635617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:49.056085 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ea98c81a-f2cf-4b17-bb50-da3ddf927994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:49.126699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1325/2645] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:49.130522 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd10548-634c-43fa-96f9-1cdd24635617 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:49.130822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98c81a-f2cf-4b17-bb50-da3ddf927994 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:49.136480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb218bbd-b514-401e-a1b1-b93e37a0c296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:49.137034 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb218bbd-b514-401e-a1b1-b93e37a0c296 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.137393 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb218bbd-b514-401e-a1b1-b93e37a0c296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.137659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb218bbd-b514-401e-a1b1-b93e37a0c296 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.138278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:49.138416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb218bbd-b514-401e-a1b1-b93e37a0c296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:49.140333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb218bbd-b514-401e-a1b1-b93e37a0c296 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:49.142447 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd10548-634c-43fa-96f9-1cdd24635617 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:49.152126 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae1801-04ba-4e79-9084-70ca87e878d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:49.153151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae1801-04ba-4e79-9084-70ca87e878d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.153704 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae1801-04ba-4e79-9084-70ca87e878d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.154389 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae1801-04ba-4e79-9084-70ca87e878d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.155045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae1801-04ba-4e79-9084-70ca87e878d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:38:49.155706 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae1801-04ba-4e79-9084-70ca87e878d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:49.157318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae1801-04ba-4e79-9084-70ca87e878d4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:49.167878 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1321/2646] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:49 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:49.172559 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb218bbd-b514-401e-a1b1-b93e37a0c296 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:49.177211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1326/2647] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:38:49.178740 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae1801-04ba-4e79-9084-70ca87e878d4 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:49.191328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb218bbd-b514-401e-a1b1-b93e37a0c296 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:49.191618 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae1801-04ba-4e79-9084-70ca87e878d4 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:49.198072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1612e-944d-4560-9971-da4f97447f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:49.198604 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b4488-35f4-4101-9bb4-fc3aeb645a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:49.199501 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1612e-944d-4560-9971-da4f97447f90 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.200156 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b4488-35f4-4101-9bb4-fc3aeb645a6e admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.200491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1612e-944d-4560-9971-da4f97447f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.200833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b4488-35f4-4101-9bb4-fc3aeb645a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.201159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1612e-944d-4560-9971-da4f97447f90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.201481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b4488-35f4-4101-9bb4-fc3aeb645a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.202174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1612e-944d-4560-9971-da4f97447f90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:49.202933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b4488-35f4-4101-9bb4-fc3aeb645a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:49.204904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1612e-944d-4560-9971-da4f97447f90 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:49.206569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b4488-35f4-4101-9bb4-fc3aeb645a6e admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:49.223963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1322/2648] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:38:49.225058 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b4488-35f4-4101-9bb4-fc3aeb645a6e admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:49.237728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b4488-35f4-4101-9bb4-fc3aeb645a6e admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:49.242532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7391ec8-ec96-4808-850a-d8aa1580459a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:49.243076 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1327/2649] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:49 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:49.246621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7391ec8-ec96-4808-850a-d8aa1580459a admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.246964 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7391ec8-ec96-4808-850a-d8aa1580459a admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.247251 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7391ec8-ec96-4808-850a-d8aa1580459a admin admin] PATH_INFO: `/v3/projects/0f9cfd46e5214dec9c779a12001a84b6/users/7b6b174c38bf43b79916c07665dac1d9/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.247508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1612e-944d-4560-9971-da4f97447f90 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:49.266433 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7391ec8-ec96-4808-850a-d8aa1580459a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f9cfd46e5214dec9c779a12001a84b6, user_id=7b6b174c38bf43b79916c07665dac1d9, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:49.269890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7391ec8-ec96-4808-850a-d8aa1580459a admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:49.284718 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1323/2650] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:49 2022] PUT /identity/v3/projects/0f9cfd46e5214dec9c779a12001a84b6/users/7b6b174c38bf43b79916c07665dac1d9/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:49.289146 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7391ec8-ec96-4808-850a-d8aa1580459a admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:49.317763 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1612e-944d-4560-9971-da4f97447f90 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:49.377718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02b4eb-d20b-46c5-9ec4-569b9633d363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:49.378356 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02b4eb-d20b-46c5-9ec4-569b9633d363 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.378707 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02b4eb-d20b-46c5-9ec4-569b9633d363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.379041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02b4eb-d20b-46c5-9ec4-569b9633d363 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.379569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7391ec8-ec96-4808-850a-d8aa1580459a admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:49.380038 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02b4eb-d20b-46c5-9ec4-569b9633d363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:49.381843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02b4eb-d20b-46c5-9ec4-569b9633d363 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:49.385924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141335c-a41e-4c41-8ac4-e787c4d697ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:49.386713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141335c-a41e-4c41-8ac4-e787c4d697ed admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.387695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141335c-a41e-4c41-8ac4-e787c4d697ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.387813 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141335c-a41e-4c41-8ac4-e787c4d697ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.389154 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3141335c-a41e-4c41-8ac4-e787c4d697ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:49.391340 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3141335c-a41e-4c41-8ac4-e787c4d697ed admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:49.404950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1324/2651] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:49.406454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1328/2652] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:49 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:49.408952 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141335c-a41e-4c41-8ac4-e787c4d697ed admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:49.410387 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02b4eb-d20b-46c5-9ec4-569b9633d363 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:49.421982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141335c-a41e-4c41-8ac4-e787c4d697ed admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:49.423921 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02b4eb-d20b-46c5-9ec4-569b9633d363 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:49.427492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c76c3-71f2-4557-b05b-349c1b779a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:49.428724 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c76c3-71f2-4557-b05b-349c1b779a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.429011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0466d-018e-4f83-a6f8-0dc6a7f7ab9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:49.429443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c76c3-71f2-4557-b05b-349c1b779a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.430035 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0466d-018e-4f83-a6f8-0dc6a7f7ab9b admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.430035 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0466d-018e-4f83-a6f8-0dc6a7f7ab9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.430371 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0466d-018e-4f83-a6f8-0dc6a7f7ab9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.430818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c76c3-71f2-4557-b05b-349c1b779a76 admin admin] PATH_INFO: `/v3/projects/0f9cfd46e5214dec9c779a12001a84b6/users/7b6b174c38bf43b79916c07665dac1d9/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.431181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0466d-018e-4f83-a6f8-0dc6a7f7ab9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:49.433159 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0466d-018e-4f83-a6f8-0dc6a7f7ab9b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:49.434144 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c76c3-71f2-4557-b05b-349c1b779a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f9cfd46e5214dec9c779a12001a84b6, user_id=7b6b174c38bf43b79916c07665dac1d9, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:49.437508 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c76c3-71f2-4557-b05b-349c1b779a76 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:49.444867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1329/2653] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:49.445507 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:49.448905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0466d-018e-4f83-a6f8-0dc6a7f7ab9b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:49.455074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1325/2654] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:49 2022] PUT /identity/v3/projects/0f9cfd46e5214dec9c779a12001a84b6/users/7b6b174c38bf43b79916c07665dac1d9/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:49.459333 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0466d-018e-4f83-a6f8-0dc6a7f7ab9b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:49.466044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d4052-1719-48d7-ad06-80022b1bd7a5 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.466403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d4052-1719-48d7-ad06-80022b1bd7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.467067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d4052-1719-48d7-ad06-80022b1bd7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.497059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-fb6d4052-1719-48d7-ad06-80022b1bd7a5 None None] MFA Rules not processed for user `7b6b174c38bf43b79916c07665dac1d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:49.498257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-fb6d4052-1719-48d7-ad06-80022b1bd7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:49.517204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:49.525708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ed6b7-7eb8-4ce1-b5ad-c693f06226d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:49.526381 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ed6b7-7eb8-4ce1-b5ad-c693f06226d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.526799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ed6b7-7eb8-4ce1-b5ad-c693f06226d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.526966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ed6b7-7eb8-4ce1-b5ad-c693f06226d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.527622 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ed6b7-7eb8-4ce1-b5ad-c693f06226d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:49.529260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ed6b7-7eb8-4ce1-b5ad-c693f06226d7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:49.539655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1330/2655] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:38:49.544105 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ed6b7-7eb8-4ce1-b5ad-c693f06226d7 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:49.555543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ed6b7-7eb8-4ce1-b5ad-c693f06226d7 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:49.560615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87701aa5-6829-4fdb-b8fb-2e7d25796ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:49.561295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87701aa5-6829-4fdb-b8fb-2e7d25796ea0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.561572 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87701aa5-6829-4fdb-b8fb-2e7d25796ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.561836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87701aa5-6829-4fdb-b8fb-2e7d25796ea0 admin admin] PATH_INFO: `/v3/projects/ff5daf9e9245447dac44278ce813b4a0/users/f86d1ec821814f628879590513702cec/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.575480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1326/2656] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:49.578991 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87701aa5-6829-4fdb-b8fb-2e7d25796ea0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff5daf9e9245447dac44278ce813b4a0, user_id=f86d1ec821814f628879590513702cec, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:49.581860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87701aa5-6829-4fdb-b8fb-2e7d25796ea0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:49.586309 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d4052-1719-48d7-ad06-80022b1bd7a5 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:49.588076 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-fb6d4052-1719-48d7-ad06-80022b1bd7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:49.595945 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1331/2657] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:49 2022] PUT /identity/v3/projects/ff5daf9e9245447dac44278ce813b4a0/users/f86d1ec821814f628879590513702cec/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:49.600920 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c49b6b-079b-40f8-bb75-070d02786e40 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.601204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c49b6b-079b-40f8-bb75-070d02786e40 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.601471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c49b6b-079b-40f8-bb75-070d02786e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.619965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c6c49b6b-079b-40f8-bb75-070d02786e40 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:49.621408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c6c49b6b-079b-40f8-bb75-070d02786e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:49.643879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d4052-1719-48d7-ad06-80022b1bd7a5 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:49.681909 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1332/2658] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:49.692243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea45f7-af81-43b0-99a2-9cd001de71e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:49.692433 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c49b6b-079b-40f8-bb75-070d02786e40 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:49.692859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea45f7-af81-43b0-99a2-9cd001de71e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.693135 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea45f7-af81-43b0-99a2-9cd001de71e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.693404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea45f7-af81-43b0-99a2-9cd001de71e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.693650 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c6c49b6b-079b-40f8-bb75-070d02786e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:49.744409 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:49.753351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c49b6b-079b-40f8-bb75-070d02786e40 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:49.757354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac18e0c-df2e-4c19-99ce-d30ca424e92e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:49.757943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac18e0c-df2e-4c19-99ce-d30ca424e92e service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:49.758218 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac18e0c-df2e-4c19-99ce-d30ca424e92e service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:49.758484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac18e0c-df2e-4c19-99ce-d30ca424e92e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:49.758612 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea45f7-af81-43b0-99a2-9cd001de71e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:49.759991 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea45f7-af81-43b0-99a2-9cd001de71e4 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:49.768021 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac18e0c-df2e-4c19-99ce-d30ca424e92e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:49.769471 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac18e0c-df2e-4c19-99ce-d30ca424e92e service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:49.773112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1327/2659] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:38:49.781338 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1333/2660] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:38:50.220521 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea45f7-af81-43b0-99a2-9cd001de71e4 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:50.252640 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea45f7-af81-43b0-99a2-9cd001de71e4 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:50.261731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05a198-f1ad-44e2-8a7d-ecc5399f824b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:50.262393 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05a198-f1ad-44e2-8a7d-ecc5399f824b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:50.262694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05a198-f1ad-44e2-8a7d-ecc5399f824b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:50.262965 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05a198-f1ad-44e2-8a7d-ecc5399f824b admin admin] PATH_INFO: `/v3/users/3fd8252e16014e7b9ccb9bfa7ab3bd6f` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:50.266603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05a198-f1ad-44e2-8a7d-ecc5399f824b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fd8252e16014e7b9ccb9bfa7ab3bd6f)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:50.268535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05a198-f1ad-44e2-8a7d-ecc5399f824b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:50.356209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ac05a198-f1ad-44e2-8a7d-ecc5399f824b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fd8252e16014e7b9ccb9bfa7ab3bd6f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:50.357085 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ac05a198-f1ad-44e2-8a7d-ecc5399f824b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fd8252e16014e7b9ccb9bfa7ab3bd6f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:50.357950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ac05a198-f1ad-44e2-8a7d-ecc5399f824b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fd8252e16014e7b9ccb9bfa7ab3bd6f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:50.368086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ac05a198-f1ad-44e2-8a7d-ecc5399f824b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fd8252e16014e7b9ccb9bfa7ab3bd6f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:50.392082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ac05a198-f1ad-44e2-8a7d-ecc5399f824b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fd8252e16014e7b9ccb9bfa7ab3bd6f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:50.411332 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1328/2661] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:50 2022] DELETE /identity/v3/users/3fd8252e16014e7b9ccb9bfa7ab3bd6f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:50.412324 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d623708-bff1-428d-b76e-1ea263bd4c23 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:50.412775 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d623708-bff1-428d-b76e-1ea263bd4c23 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:50.413339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d623708-bff1-428d-b76e-1ea263bd4c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:50.449009 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-8d623708-bff1-428d-b76e-1ea263bd4c23 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:50.450467 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8d623708-bff1-428d-b76e-1ea263bd4c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:50.518628 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1334/2662] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:50.526344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05a198-f1ad-44e2-8a7d-ecc5399f824b admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:50.527872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ac05a198-f1ad-44e2-8a7d-ecc5399f824b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:50.599724 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05a198-f1ad-44e2-8a7d-ecc5399f824b admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:50.603359 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64819417-674b-4f42-b5c7-02a2857a120c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:50.604010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64819417-674b-4f42-b5c7-02a2857a120c service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:50.604288 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64819417-674b-4f42-b5c7-02a2857a120c service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:50.604560 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64819417-674b-4f42-b5c7-02a2857a120c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:50.613692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64819417-674b-4f42-b5c7-02a2857a120c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:50.615329 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64819417-674b-4f42-b5c7-02a2857a120c service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:50.625397 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1329/2663] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:50.899012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d623708-bff1-428d-b76e-1ea263bd4c23 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:50.900943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8d623708-bff1-428d-b76e-1ea263bd4c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:50.914432 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d623708-bff1-428d-b76e-1ea263bd4c23 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:50.919865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43609cfe-43c3-4aa4-848e-660d15357b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:50.920633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43609cfe-43c3-4aa4-848e-660d15357b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:50.920968 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43609cfe-43c3-4aa4-848e-660d15357b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:50.921318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43609cfe-43c3-4aa4-848e-660d15357b39 admin admin] PATH_INFO: `/v3/projects/550daf0d9c1a4920aeb2fe44a9cfe10a` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:50.922684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43609cfe-43c3-4aa4-848e-660d15357b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=550daf0d9c1a4920aeb2fe44a9cfe10a)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:50.924797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43609cfe-43c3-4aa4-848e-660d15357b39 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:50.932452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:51.001925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-43609cfe-43c3-4aa4-848e-660d15357b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '550daf0d9c1a4920aeb2fe44a9cfe10a', 'request_id': 'req-43609cfe-43c3-4aa4-848e-660d15357b39'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:51.014355 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-43609cfe-43c3-4aa4-848e-660d15357b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '550daf0d9c1a4920aeb2fe44a9cfe10a', 'request_id': 'req-43609cfe-43c3-4aa4-848e-660d15357b39'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:51.023991 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-43609cfe-43c3-4aa4-848e-660d15357b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '550daf0d9c1a4920aeb2fe44a9cfe10a', 'request_id': 'req-43609cfe-43c3-4aa4-848e-660d15357b39'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:51.041042 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-43609cfe-43c3-4aa4-848e-660d15357b39 admin admin] The token cache is being invalidate because project 550daf0d9c1a4920aeb2fe44a9cfe10a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:51.041332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-43609cfe-43c3-4aa4-848e-660d15357b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:51.042081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-43609cfe-43c3-4aa4-848e-660d15357b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:51.046796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1335/2664] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:50 2022] DELETE /identity/v3/projects/550daf0d9c1a4920aeb2fe44a9cfe10a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:51.048149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64819417-674b-4f42-b5c7-02a2857a120c service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:51.062534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64819417-674b-4f42-b5c7-02a2857a120c service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:51.152863 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f0063-6223-4b18-9271-d95fc7f76a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:51.153728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f0063-6223-4b18-9271-d95fc7f76a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:51.154172 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f0063-6223-4b18-9271-d95fc7f76a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:51.154642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f0063-6223-4b18-9271-d95fc7f76a1b admin admin] PATH_INFO: `/v3/users/7b6b174c38bf43b79916c07665dac1d9` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:51.155940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f0063-6223-4b18-9271-d95fc7f76a1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b6b174c38bf43b79916c07665dac1d9)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:51.160462 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f0063-6223-4b18-9271-d95fc7f76a1b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:51.168189 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aec086-5dae-4318-b82f-9ce35eaf100e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:51.168876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aec086-5dae-4318-b82f-9ce35eaf100e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:51.169321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aec086-5dae-4318-b82f-9ce35eaf100e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:51.176155 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:51.193967 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-78aec086-5dae-4318-b82f-9ce35eaf100e None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:51.195582 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-78aec086-5dae-4318-b82f-9ce35eaf100e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:51.250987 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-fe6f0063-6223-4b18-9271-d95fc7f76a1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b6b174c38bf43b79916c07665dac1d9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:51.252480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-fe6f0063-6223-4b18-9271-d95fc7f76a1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b6b174c38bf43b79916c07665dac1d9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:51.253472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-fe6f0063-6223-4b18-9271-d95fc7f76a1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b6b174c38bf43b79916c07665dac1d9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:51.268883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-fe6f0063-6223-4b18-9271-d95fc7f76a1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b6b174c38bf43b79916c07665dac1d9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:51.280220 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1336/2665] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:38:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:51.295327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-fe6f0063-6223-4b18-9271-d95fc7f76a1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b6b174c38bf43b79916c07665dac1d9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:51.297445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aec086-5dae-4318-b82f-9ce35eaf100e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:51.299796 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-78aec086-5dae-4318-b82f-9ce35eaf100e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:51.318208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1330/2666] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:51 2022] DELETE /identity/v3/users/7b6b174c38bf43b79916c07665dac1d9 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:51.445490 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aec086-5dae-4318-b82f-9ce35eaf100e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:51.512526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:51.543837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f40af-f24e-497e-b3c1-e608441ffa81 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:51.545041 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f40af-f24e-497e-b3c1-e608441ffa81 service swift] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:51.545419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f40af-f24e-497e-b3c1-e608441ffa81 service swift] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:51.545820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f40af-f24e-497e-b3c1-e608441ffa81 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:51.636474 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f40af-f24e-497e-b3c1-e608441ffa81 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:51.638969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f40af-f24e-497e-b3c1-e608441ffa81 service swift] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:51.649868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1337/2667] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:38:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 16:38:52.192525 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f0063-6223-4b18-9271-d95fc7f76a1b admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:52.195184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-fe6f0063-6223-4b18-9271-d95fc7f76a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:52.322982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f0063-6223-4b18-9271-d95fc7f76a1b admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:52.328399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f64289-cf6d-4993-8513-5dea2d49201d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:52.329281 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64289-cf6d-4993-8513-5dea2d49201d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:52.329719 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64289-cf6d-4993-8513-5dea2d49201d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:52.330268 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64289-cf6d-4993-8513-5dea2d49201d admin admin] PATH_INFO: `/v3/projects/0f9cfd46e5214dec9c779a12001a84b6` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:52.331796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f64289-cf6d-4993-8513-5dea2d49201d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f9cfd46e5214dec9c779a12001a84b6)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:52.333938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f64289-cf6d-4993-8513-5dea2d49201d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:52.397933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-72f64289-cf6d-4993-8513-5dea2d49201d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f9cfd46e5214dec9c779a12001a84b6', 'request_id': 'req-72f64289-cf6d-4993-8513-5dea2d49201d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:52.410734 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-72f64289-cf6d-4993-8513-5dea2d49201d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f9cfd46e5214dec9c779a12001a84b6', 'request_id': 'req-72f64289-cf6d-4993-8513-5dea2d49201d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:52.436477 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-72f64289-cf6d-4993-8513-5dea2d49201d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f9cfd46e5214dec9c779a12001a84b6', 'request_id': 'req-72f64289-cf6d-4993-8513-5dea2d49201d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:52.464841 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-72f64289-cf6d-4993-8513-5dea2d49201d admin admin] The token cache is being invalidate because project 0f9cfd46e5214dec9c779a12001a84b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:52.465150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-72f64289-cf6d-4993-8513-5dea2d49201d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:52.465888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-72f64289-cf6d-4993-8513-5dea2d49201d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:52.481305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1331/2668] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:52 2022] DELETE /identity/v3/projects/0f9cfd46e5214dec9c779a12001a84b6 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:52.484801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8c26a-489d-4308-8dff-015e8bab402c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:52.485253 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8c26a-489d-4308-8dff-015e8bab402c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:52.485493 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8c26a-489d-4308-8dff-015e8bab402c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:52.516101 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-70a8c26a-489d-4308-8dff-015e8bab402c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:52.517419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-70a8c26a-489d-4308-8dff-015e8bab402c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:52.660141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1338/2669] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:52.669888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a2f42-9ab1-4f30-8ee3-50c8a8fc0b1a None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:52.670275 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a2f42-9ab1-4f30-8ee3-50c8a8fc0b1a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:52.670670 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a2f42-9ab1-4f30-8ee3-50c8a8fc0b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:52.710828 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-cd7a2f42-9ab1-4f30-8ee3-50c8a8fc0b1a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:52.713812 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cd7a2f42-9ab1-4f30-8ee3-50c8a8fc0b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:52.738277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1332/2670] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:52.742629 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8c26a-489d-4308-8dff-015e8bab402c None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:52.758215 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8c26a-489d-4308-8dff-015e8bab402c None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:52.774351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a6a87-ca82-4a65-9ded-3dd9e90818b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:52.775013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a6a87-ca82-4a65-9ded-3dd9e90818b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:52.775322 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a6a87-ca82-4a65-9ded-3dd9e90818b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:52.775554 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a6a87-ca82-4a65-9ded-3dd9e90818b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:52.776391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a6a87-ca82-4a65-9ded-3dd9e90818b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:38:52.776697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a6a87-ca82-4a65-9ded-3dd9e90818b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:52.779640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a6a87-ca82-4a65-9ded-3dd9e90818b8 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:52.805142 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1339/2671] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:38:52.809905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a2f42-9ab1-4f30-8ee3-50c8a8fc0b1a None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:52.820372 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a2f42-9ab1-4f30-8ee3-50c8a8fc0b1a None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:52.834694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91cfa7-e826-4606-817f-5b6d194e0a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:52.835339 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91cfa7-e826-4606-817f-5b6d194e0a04 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:52.835615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91cfa7-e826-4606-817f-5b6d194e0a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:52.835883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91cfa7-e826-4606-817f-5b6d194e0a04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:52.836733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91cfa7-e826-4606-817f-5b6d194e0a04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:52.844749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91cfa7-e826-4606-817f-5b6d194e0a04 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:52.863967 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:52.914134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1333/2672] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:52 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:52.918384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a6a87-ca82-4a65-9ded-3dd9e90818b8 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:52.932678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a6a87-ca82-4a65-9ded-3dd9e90818b8 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:53.140891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f568284-9af0-49da-8383-1eefbbf4b2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:53.141817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f568284-9af0-49da-8383-1eefbbf4b2a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:53.142248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f568284-9af0-49da-8383-1eefbbf4b2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:53.142693 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f568284-9af0-49da-8383-1eefbbf4b2a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:53.143467 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f568284-9af0-49da-8383-1eefbbf4b2a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:53.145594 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f568284-9af0-49da-8383-1eefbbf4b2a9 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:53.190559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:53.198873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1340/2673] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:52 2022] POST /identity/v3/users => generated 291 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:53.201107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91cfa7-e826-4606-817f-5b6d194e0a04 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:53.217248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91cfa7-e826-4606-817f-5b6d194e0a04 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:53.230817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57e3fe-a201-4d54-8d0d-a9a46ae39154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:53.231891 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57e3fe-a201-4d54-8d0d-a9a46ae39154 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:53.232346 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57e3fe-a201-4d54-8d0d-a9a46ae39154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:53.232793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57e3fe-a201-4d54-8d0d-a9a46ae39154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:53.233591 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57e3fe-a201-4d54-8d0d-a9a46ae39154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:53.235403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57e3fe-a201-4d54-8d0d-a9a46ae39154 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:53.262846 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1334/2674] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:53.265035 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f568284-9af0-49da-8383-1eefbbf4b2a9 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:53.292979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f568284-9af0-49da-8383-1eefbbf4b2a9 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:53.318396 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966e062-69b9-48d8-883f-79da5e217713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:53.319273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966e062-69b9-48d8-883f-79da5e217713 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:53.319692 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966e062-69b9-48d8-883f-79da5e217713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:53.320138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966e062-69b9-48d8-883f-79da5e217713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:53.320941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966e062-69b9-48d8-883f-79da5e217713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:53.322833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966e062-69b9-48d8-883f-79da5e217713 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:53.362740 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1341/2675] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:38:53.377099 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57e3fe-a201-4d54-8d0d-a9a46ae39154 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:53.390463 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57e3fe-a201-4d54-8d0d-a9a46ae39154 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:53.402250 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de080da-4634-440c-a5c4-0f0536b1c6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:53.407007 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de080da-4634-440c-a5c4-0f0536b1c6d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:53.407291 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de080da-4634-440c-a5c4-0f0536b1c6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:53.407567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de080da-4634-440c-a5c4-0f0536b1c6d2 admin admin] PATH_INFO: `/v3/projects/1bc2500a995646ccbd068983290b6cd9/users/c018eaadff184228bb6bd379cbf225d3/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:53.425959 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de080da-4634-440c-a5c4-0f0536b1c6d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc2500a995646ccbd068983290b6cd9, user_id=c018eaadff184228bb6bd379cbf225d3, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:53.446849 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de080da-4634-440c-a5c4-0f0536b1c6d2 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:53.467866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1335/2676] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:53 2022] PUT /identity/v3/projects/1bc2500a995646ccbd068983290b6cd9/users/c018eaadff184228bb6bd379cbf225d3/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:53.479214 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a77600-a127-4ec3-8b65-2e7d99b1644e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:53.479865 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a77600-a127-4ec3-8b65-2e7d99b1644e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:53.480313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a77600-a127-4ec3-8b65-2e7d99b1644e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:53.544063 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-f8a77600-a127-4ec3-8b65-2e7d99b1644e None None] MFA Rules not processed for user `c018eaadff184228bb6bd379cbf225d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:53.545840 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f8a77600-a127-4ec3-8b65-2e7d99b1644e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:53.708892 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1342/2677] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:53.714028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de080da-4634-440c-a5c4-0f0536b1c6d2 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:53.845495 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de080da-4634-440c-a5c4-0f0536b1c6d2 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:53.851012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480ee89-4dea-4e80-9a99-c77ff1638995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:53.851895 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480ee89-4dea-4e80-9a99-c77ff1638995 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:53.852399 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480ee89-4dea-4e80-9a99-c77ff1638995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:53.852873 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480ee89-4dea-4e80-9a99-c77ff1638995 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:53.853726 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6480ee89-4dea-4e80-9a99-c77ff1638995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:53.856023 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6480ee89-4dea-4e80-9a99-c77ff1638995 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:53.925230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a77600-a127-4ec3-8b65-2e7d99b1644e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:53.926563 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1336/2678] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:53 2022] POST /identity/v3/projects => generated 381 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:53.940484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a77600-a127-4ec3-8b65-2e7d99b1644e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:54.057213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab989e7-b2f7-4883-b71f-c547b9029293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:54.057844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab989e7-b2f7-4883-b71f-c547b9029293 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:54.058115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab989e7-b2f7-4883-b71f-c547b9029293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:54.058382 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab989e7-b2f7-4883-b71f-c547b9029293 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:54.059151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab989e7-b2f7-4883-b71f-c547b9029293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:54.061163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab989e7-b2f7-4883-b71f-c547b9029293 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:54.158092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1343/2679] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:53 2022] POST /identity/v3/users => generated 297 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:54.162519 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480ee89-4dea-4e80-9a99-c77ff1638995 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:54.181733 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480ee89-4dea-4e80-9a99-c77ff1638995 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:54.189152 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fbba7-369c-44c4-8290-c7f121d312c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:54.190148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fbba7-369c-44c4-8290-c7f121d312c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:54.190674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fbba7-369c-44c4-8290-c7f121d312c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:54.191123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fbba7-369c-44c4-8290-c7f121d312c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:54.193403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fbba7-369c-44c4-8290-c7f121d312c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:54.195436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fbba7-369c-44c4-8290-c7f121d312c2 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:54.213877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1337/2680] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:54.214594 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab989e7-b2f7-4883-b71f-c547b9029293 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:54.242033 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab989e7-b2f7-4883-b71f-c547b9029293 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:54.255092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a029d41-b2a8-4817-bc08-66b722e76b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:54.261767 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a029d41-b2a8-4817-bc08-66b722e76b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:54.262066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a029d41-b2a8-4817-bc08-66b722e76b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:54.262332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a029d41-b2a8-4817-bc08-66b722e76b0f admin admin] PATH_INFO: `/v3/projects/f9721c68ade7464ca98112b7dfc254e5/users/51d57573d3bb4ae9ac5885f7dafe7a7d/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:54.312315 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a029d41-b2a8-4817-bc08-66b722e76b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9721c68ade7464ca98112b7dfc254e5, user_id=51d57573d3bb4ae9ac5885f7dafe7a7d, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:54.315137 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a029d41-b2a8-4817-bc08-66b722e76b0f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:54.367074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1344/2681] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:54 2022] PUT /identity/v3/projects/f9721c68ade7464ca98112b7dfc254e5/users/51d57573d3bb4ae9ac5885f7dafe7a7d/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:54.371200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fbba7-369c-44c4-8290-c7f121d312c2 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:54.386354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fbba7-369c-44c4-8290-c7f121d312c2 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:54.548900 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ff373-f3b4-4625-8b09-903ab2f9059f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:54.550132 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ff373-f3b4-4625-8b09-903ab2f9059f admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:54.550416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ff373-f3b4-4625-8b09-903ab2f9059f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:54.550856 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ff373-f3b4-4625-8b09-903ab2f9059f admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:54.551436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ff373-f3b4-4625-8b09-903ab2f9059f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:54.553384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ff373-f3b4-4625-8b09-903ab2f9059f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:54.576579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1338/2682] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:54.580208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a029d41-b2a8-4817-bc08-66b722e76b0f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:54.597026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a029d41-b2a8-4817-bc08-66b722e76b0f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:54.620904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc52b4-c979-4495-b597-a4e8839438b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:54.621593 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc52b4-c979-4495-b597-a4e8839438b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:54.621882 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc52b4-c979-4495-b597-a4e8839438b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:54.622163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc52b4-c979-4495-b597-a4e8839438b2 admin admin] PATH_INFO: `/v3/projects/f9721c68ade7464ca98112b7dfc254e5/users/51d57573d3bb4ae9ac5885f7dafe7a7d/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:54.630205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc52b4-c979-4495-b597-a4e8839438b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9721c68ade7464ca98112b7dfc254e5, user_id=51d57573d3bb4ae9ac5885f7dafe7a7d, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:54.636104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc52b4-c979-4495-b597-a4e8839438b2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:54.667775 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1345/2683] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:54 2022] PUT /identity/v3/projects/f9721c68ade7464ca98112b7dfc254e5/users/51d57573d3bb4ae9ac5885f7dafe7a7d/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:54.679139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df0db7-0d0a-46e7-ab2d-907141fb622f None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:54.679642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df0db7-0d0a-46e7-ab2d-907141fb622f None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:54.679921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df0db7-0d0a-46e7-ab2d-907141fb622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:54.740296 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d5df0db7-0d0a-46e7-ab2d-907141fb622f None None] MFA Rules not processed for user `51d57573d3bb4ae9ac5885f7dafe7a7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:54.741557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d5df0db7-0d0a-46e7-ab2d-907141fb622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:54.760582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:54.889107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1339/2684] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:54.904393 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc52b4-c979-4495-b597-a4e8839438b2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:54.906133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e6cc52b4-c979-4495-b597-a4e8839438b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:54.983384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:55.029332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc52b4-c979-4495-b597-a4e8839438b2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:55.043113 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84878075-971e-43b9-af76-a5c22e988ede service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:55.043747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84878075-971e-43b9-af76-a5c22e988ede service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:55.044024 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84878075-971e-43b9-af76-a5c22e988ede service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:55.044398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84878075-971e-43b9-af76-a5c22e988ede service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:55.070761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84878075-971e-43b9-af76-a5c22e988ede service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:55.072245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84878075-971e-43b9-af76-a5c22e988ede service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:55.083398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1346/2685] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:55.198134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df0db7-0d0a-46e7-ab2d-907141fb622f None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:55.210369 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df0db7-0d0a-46e7-ab2d-907141fb622f None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:55.224277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62efcb7-f5ec-4320-83dc-3f6abc399ef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:55.224951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62efcb7-f5ec-4320-83dc-3f6abc399ef5 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:55.225187 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62efcb7-f5ec-4320-83dc-3f6abc399ef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:55.225457 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62efcb7-f5ec-4320-83dc-3f6abc399ef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:55.334203 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62efcb7-f5ec-4320-83dc-3f6abc399ef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:55.335883 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62efcb7-f5ec-4320-83dc-3f6abc399ef5 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:55.363032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1340/2686] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:55.859505 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84878075-971e-43b9-af76-a5c22e988ede service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:55.992499 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84878075-971e-43b9-af76-a5c22e988ede service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:56.002709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb7f29-7fe6-4b57-896b-1240b9377d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:56.003912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb7f29-7fe6-4b57-896b-1240b9377d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:56.003912 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb7f29-7fe6-4b57-896b-1240b9377d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:56.004126 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb7f29-7fe6-4b57-896b-1240b9377d9d admin admin] PATH_INFO: `/v3/users/c018eaadff184228bb6bd379cbf225d3` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:56.005846 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb7f29-7fe6-4b57-896b-1240b9377d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c018eaadff184228bb6bd379cbf225d3)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:56.009898 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb7f29-7fe6-4b57-896b-1240b9377d9d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:56.137202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-47eb7f29-7fe6-4b57-896b-1240b9377d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c018eaadff184228bb6bd379cbf225d3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:56.141310 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-47eb7f29-7fe6-4b57-896b-1240b9377d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c018eaadff184228bb6bd379cbf225d3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:56.143644 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-47eb7f29-7fe6-4b57-896b-1240b9377d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c018eaadff184228bb6bd379cbf225d3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:56.170535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-47eb7f29-7fe6-4b57-896b-1240b9377d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c018eaadff184228bb6bd379cbf225d3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:56.193521 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-47eb7f29-7fe6-4b57-896b-1240b9377d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c018eaadff184228bb6bd379cbf225d3'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:56.223891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1347/2687] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:55 2022] DELETE /identity/v3/users/c018eaadff184228bb6bd379cbf225d3 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:56.225529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5eb578-d41a-4890-9a05-a9331000035d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:56.226118 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5eb578-d41a-4890-9a05-a9331000035d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:56.226118 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5eb578-d41a-4890-9a05-a9331000035d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:56.251288 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-2d5eb578-d41a-4890-9a05-a9331000035d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:56.253331 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2d5eb578-d41a-4890-9a05-a9331000035d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:56.374673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1341/2688] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:38:56.381927 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb7f29-7fe6-4b57-896b-1240b9377d9d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:56.383836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-47eb7f29-7fe6-4b57-896b-1240b9377d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:56.486337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb7f29-7fe6-4b57-896b-1240b9377d9d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:56.491583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf84d8-2cf5-4520-a8a6-55adbb2c8fb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:56.491940 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf84d8-2cf5-4520-a8a6-55adbb2c8fb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:56.492309 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf84d8-2cf5-4520-a8a6-55adbb2c8fb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:56.492641 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf84d8-2cf5-4520-a8a6-55adbb2c8fb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:56.521236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:56.531588 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf84d8-2cf5-4520-a8a6-55adbb2c8fb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:56.535364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf84d8-2cf5-4520-a8a6-55adbb2c8fb9 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:56.560491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1348/2689] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:38:57.088734 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5eb578-d41a-4890-9a05-a9331000035d None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:57.090907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2d5eb578-d41a-4890-9a05-a9331000035d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:57.113051 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5eb578-d41a-4890-9a05-a9331000035d None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:57.129841 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822d31d-ea24-42a3-a8ef-514a8f71447c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:57.131556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822d31d-ea24-42a3-a8ef-514a8f71447c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:57.131949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822d31d-ea24-42a3-a8ef-514a8f71447c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:57.132474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822d31d-ea24-42a3-a8ef-514a8f71447c admin admin] PATH_INFO: `/v3/projects/1bc2500a995646ccbd068983290b6cd9` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:57.133963 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822d31d-ea24-42a3-a8ef-514a8f71447c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc2500a995646ccbd068983290b6cd9)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:57.140446 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822d31d-ea24-42a3-a8ef-514a8f71447c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:57.300561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:57.322037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c822d31d-ea24-42a3-a8ef-514a8f71447c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc2500a995646ccbd068983290b6cd9', 'request_id': 'req-c822d31d-ea24-42a3-a8ef-514a8f71447c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:57.340252 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c822d31d-ea24-42a3-a8ef-514a8f71447c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc2500a995646ccbd068983290b6cd9', 'request_id': 'req-c822d31d-ea24-42a3-a8ef-514a8f71447c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:57.366839 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c822d31d-ea24-42a3-a8ef-514a8f71447c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc2500a995646ccbd068983290b6cd9', 'request_id': 'req-c822d31d-ea24-42a3-a8ef-514a8f71447c'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:57.380649 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c822d31d-ea24-42a3-a8ef-514a8f71447c admin admin] The token cache is being invalidate because project 1bc2500a995646ccbd068983290b6cd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:57.381029 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c822d31d-ea24-42a3-a8ef-514a8f71447c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:57.381631 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-c822d31d-ea24-42a3-a8ef-514a8f71447c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:57.383771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1342/2690] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:57 2022] DELETE /identity/v3/projects/1bc2500a995646ccbd068983290b6cd9 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:57.390740 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf84d8-2cf5-4520-a8a6-55adbb2c8fb9 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:57.403419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf84d8-2cf5-4520-a8a6-55adbb2c8fb9 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:57.561733 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617b1ea-fa49-4e7f-94ca-bd221c837bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:57.562438 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617b1ea-fa49-4e7f-94ca-bd221c837bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:57.562764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617b1ea-fa49-4e7f-94ca-bd221c837bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:57.563031 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617b1ea-fa49-4e7f-94ca-bd221c837bbc admin admin] PATH_INFO: `/v3/users/51d57573d3bb4ae9ac5885f7dafe7a7d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:57.565342 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617b1ea-fa49-4e7f-94ca-bd221c837bbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=51d57573d3bb4ae9ac5885f7dafe7a7d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:57.570529 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617b1ea-fa49-4e7f-94ca-bd221c837bbc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:57.593178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92f45c-ec5d-4dae-a55b-c98a85918a61 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:57.593791 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92f45c-ec5d-4dae-a55b-c98a85918a61 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:57.594062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92f45c-ec5d-4dae-a55b-c98a85918a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:57.645323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-5f92f45c-ec5d-4dae-a55b-c98a85918a61 None None] MFA Rules not processed for user `f86d1ec821814f628879590513702cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:57.647859 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5f92f45c-ec5d-4dae-a55b-c98a85918a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:57.667384 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3617b1ea-fa49-4e7f-94ca-bd221c837bbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51d57573d3bb4ae9ac5885f7dafe7a7d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:57.668132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3617b1ea-fa49-4e7f-94ca-bd221c837bbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51d57573d3bb4ae9ac5885f7dafe7a7d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:57.668797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3617b1ea-fa49-4e7f-94ca-bd221c837bbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51d57573d3bb4ae9ac5885f7dafe7a7d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:57.691153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3617b1ea-fa49-4e7f-94ca-bd221c837bbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51d57573d3bb4ae9ac5885f7dafe7a7d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:57.724860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3617b1ea-fa49-4e7f-94ca-bd221c837bbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51d57573d3bb4ae9ac5885f7dafe7a7d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:57.763511 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1349/2691] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:38:57 2022] DELETE /identity/v3/users/51d57573d3bb4ae9ac5885f7dafe7a7d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:57.794046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1343/2692] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:57.804002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617b1ea-fa49-4e7f-94ca-bd221c837bbc admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:57.805661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3617b1ea-fa49-4e7f-94ca-bd221c837bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:58.021908 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617b1ea-fa49-4e7f-94ca-bd221c837bbc admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:58.050573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89700228-c9f2-4a01-8c11-94525c6a4f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:58.051517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89700228-c9f2-4a01-8c11-94525c6a4f69 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:58.051795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89700228-c9f2-4a01-8c11-94525c6a4f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:58.052066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89700228-c9f2-4a01-8c11-94525c6a4f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:58.052710 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89700228-c9f2-4a01-8c11-94525c6a4f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:58.054698 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89700228-c9f2-4a01-8c11-94525c6a4f69 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:58.125824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:58.139571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1350/2693] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:57 2022] POST /identity/v3/projects => generated 383 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:58.143657 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92f45c-ec5d-4dae-a55b-c98a85918a61 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:58.166659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92f45c-ec5d-4dae-a55b-c98a85918a61 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:58.327663 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91baba7a-e7e8-459e-a613-d090ae9fcae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:58.332448 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91baba7a-e7e8-459e-a613-d090ae9fcae8 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:58.332731 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91baba7a-e7e8-459e-a613-d090ae9fcae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:58.333004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91baba7a-e7e8-459e-a613-d090ae9fcae8 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:58.335789 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91baba7a-e7e8-459e-a613-d090ae9fcae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:58.341997 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91baba7a-e7e8-459e-a613-d090ae9fcae8 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:58.388217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1344/2694] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:58 2022] POST /identity/v3/users => generated 298 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:38:58.392256 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89700228-c9f2-4a01-8c11-94525c6a4f69 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:58.403838 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89700228-c9f2-4a01-8c11-94525c6a4f69 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:58.427677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190dfe6-1927-4003-980e-bf3bb5c015a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:58.428328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190dfe6-1927-4003-980e-bf3bb5c015a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:58.428609 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190dfe6-1927-4003-980e-bf3bb5c015a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:58.428884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190dfe6-1927-4003-980e-bf3bb5c015a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:58.429548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190dfe6-1927-4003-980e-bf3bb5c015a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:58.431208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190dfe6-1927-4003-980e-bf3bb5c015a2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:58.462608 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91baba7a-e7e8-459e-a613-d090ae9fcae8 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:58.462872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1351/2695] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:58.472611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91baba7a-e7e8-459e-a613-d090ae9fcae8 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:58.481597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ce3eb-ebfa-4adb-8a6e-1cd17f1fa396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:58.481597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ce3eb-ebfa-4adb-8a6e-1cd17f1fa396 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:58.481597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ce3eb-ebfa-4adb-8a6e-1cd17f1fa396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:58.481597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ce3eb-ebfa-4adb-8a6e-1cd17f1fa396 admin admin] PATH_INFO: `/v3/projects/a95b4d8c1df849c68e45fe23c0eca31a/users/5eb42a654af74ea59f0bd614db3c3c0d/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:58.482699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190dfe6-1927-4003-980e-bf3bb5c015a2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:58.484406 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2190dfe6-1927-4003-980e-bf3bb5c015a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:58.508478 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190dfe6-1927-4003-980e-bf3bb5c015a2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:58.540686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dce7e-a85b-42b3-83d5-b5d0c466887b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:58.541143 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dce7e-a85b-42b3-83d5-b5d0c466887b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:58.541510 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dce7e-a85b-42b3-83d5-b5d0c466887b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:58.541825 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dce7e-a85b-42b3-83d5-b5d0c466887b admin admin] PATH_INFO: `/v3/projects/f9721c68ade7464ca98112b7dfc254e5` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:58.545184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ce3eb-ebfa-4adb-8a6e-1cd17f1fa396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a95b4d8c1df849c68e45fe23c0eca31a, user_id=5eb42a654af74ea59f0bd614db3c3c0d, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:58.545535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321dce7e-a85b-42b3-83d5-b5d0c466887b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9721c68ade7464ca98112b7dfc254e5)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:58.547975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ce3eb-ebfa-4adb-8a6e-1cd17f1fa396 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:58.548184 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321dce7e-a85b-42b3-83d5-b5d0c466887b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:58.567169 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1345/2696] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:58 2022] PUT /identity/v3/projects/a95b4d8c1df849c68e45fe23c0eca31a/users/5eb42a654af74ea59f0bd614db3c3c0d/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:58.571582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ce3eb-ebfa-4adb-8a6e-1cd17f1fa396 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:58.599518 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:58.623261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-321dce7e-a85b-42b3-83d5-b5d0c466887b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9721c68ade7464ca98112b7dfc254e5', 'request_id': 'req-321dce7e-a85b-42b3-83d5-b5d0c466887b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:58.630403 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-321dce7e-a85b-42b3-83d5-b5d0c466887b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9721c68ade7464ca98112b7dfc254e5', 'request_id': 'req-321dce7e-a85b-42b3-83d5-b5d0c466887b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:58.637617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-321dce7e-a85b-42b3-83d5-b5d0c466887b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9721c68ade7464ca98112b7dfc254e5', 'request_id': 'req-321dce7e-a85b-42b3-83d5-b5d0c466887b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:58.652148 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ce3eb-ebfa-4adb-8a6e-1cd17f1fa396 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:58.652948 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-321dce7e-a85b-42b3-83d5-b5d0c466887b admin admin] The token cache is being invalidate because project f9721c68ade7464ca98112b7dfc254e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:38:58.653305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-321dce7e-a85b-42b3-83d5-b5d0c466887b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:58.654008 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-321dce7e-a85b-42b3-83d5-b5d0c466887b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:38:58.655955 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1352/2697] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:38:58 2022] DELETE /identity/v3/projects/f9721c68ade7464ca98112b7dfc254e5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:38:58.661089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353a648-0841-4ffe-bb7f-5ba9d747fc8c None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:58.661369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353a648-0841-4ffe-bb7f-5ba9d747fc8c None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:58.661825 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353a648-0841-4ffe-bb7f-5ba9d747fc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:58.684786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a353a648-0841-4ffe-bb7f-5ba9d747fc8c None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:58.686149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a353a648-0841-4ffe-bb7f-5ba9d747fc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:58.732246 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f01d2c-d5bc-4802-9a02-245fc88c96f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:58.733288 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f01d2c-d5bc-4802-9a02-245fc88c96f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:58.733794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f01d2c-d5bc-4802-9a02-245fc88c96f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:58.739016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f01d2c-d5bc-4802-9a02-245fc88c96f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:58.739722 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f01d2c-d5bc-4802-9a02-245fc88c96f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:58.741484 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f01d2c-d5bc-4802-9a02-245fc88c96f9 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:58.761210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1353/2698] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:58.764897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc586b7d-0ce7-461b-b68d-730e3c7513f8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:58.764897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc586b7d-0ce7-461b-b68d-730e3c7513f8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:58.764897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc586b7d-0ce7-461b-b68d-730e3c7513f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:58.777975 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1346/2699] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:58.778982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f01d2c-d5bc-4802-9a02-245fc88c96f9 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:58.784524 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-77f01d2c-d5bc-4802-9a02-245fc88c96f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:58.793277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-dc586b7d-0ce7-461b-b68d-730e3c7513f8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:58.794602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dc586b7d-0ce7-461b-b68d-730e3c7513f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:58.804032 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f01d2c-d5bc-4802-9a02-245fc88c96f9 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:58.811819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1354/2700] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:58.813769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9bbb5-5a65-4f8e-9628-e9857491bedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:58.815070 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9bbb5-5a65-4f8e-9628-e9857491bedd admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:58.815613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9bbb5-5a65-4f8e-9628-e9857491bedd admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:58.816164 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9bbb5-5a65-4f8e-9628-e9857491bedd admin admin] PATH_INFO: `/v3/projects/a95b4d8c1df849c68e45fe23c0eca31a/users/5eb42a654af74ea59f0bd614db3c3c0d/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:58.817615 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc586b7d-0ce7-461b-b68d-730e3c7513f8 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:58.819171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc9bbb5-5a65-4f8e-9628-e9857491bedd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a95b4d8c1df849c68e45fe23c0eca31a, user_id=5eb42a654af74ea59f0bd614db3c3c0d, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:58.826041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc9bbb5-5a65-4f8e-9628-e9857491bedd admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:58.833741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc586b7d-0ce7-461b-b68d-730e3c7513f8 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:58.838785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ec933-9b50-4de6-a1c0-00154f9a5683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:58.839855 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ec933-9b50-4de6-a1c0-00154f9a5683 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:58.840134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ec933-9b50-4de6-a1c0-00154f9a5683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:58.840452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ec933-9b50-4de6-a1c0-00154f9a5683 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:58.841156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ec933-9b50-4de6-a1c0-00154f9a5683 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:38:58.841443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ec933-9b50-4de6-a1c0-00154f9a5683 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:58.843916 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ec933-9b50-4de6-a1c0-00154f9a5683 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:58.854044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1347/2701] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:58 2022] PUT /identity/v3/projects/a95b4d8c1df849c68e45fe23c0eca31a/users/5eb42a654af74ea59f0bd614db3c3c0d/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:58.858813 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1355/2702] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:38:58.863016 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9bbb5-5a65-4f8e-9628-e9857491bedd admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:59.006851 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9bbb5-5a65-4f8e-9628-e9857491bedd admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:59.012435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a23867-bd52-4ded-823a-2ec358ceef9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:59.013358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a23867-bd52-4ded-823a-2ec358ceef9c admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:59.013868 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a23867-bd52-4ded-823a-2ec358ceef9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:59.014338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a23867-bd52-4ded-823a-2ec358ceef9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:59.015414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a23867-bd52-4ded-823a-2ec358ceef9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:59.049549 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a23867-bd52-4ded-823a-2ec358ceef9c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:59.107031 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1348/2703] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:38:58 2022] POST /identity/v3/projects => generated 375 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:59.108391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ec933-9b50-4de6-a1c0-00154f9a5683 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:59.128483 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ec933-9b50-4de6-a1c0-00154f9a5683 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:59.214612 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:59.228219 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2c9e3-5923-4122-a7cd-0c732cab6ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:59.229208 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2c9e3-5923-4122-a7cd-0c732cab6ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:59.229679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2c9e3-5923-4122-a7cd-0c732cab6ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:59.230181 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2c9e3-5923-4122-a7cd-0c732cab6ab2 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:59.231034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2c9e3-5923-4122-a7cd-0c732cab6ab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:59.233244 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2c9e3-5923-4122-a7cd-0c732cab6ab2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:59.265441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1356/2704] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:38:59 2022] POST /identity/v3/users => generated 288 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:38:59.269508 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a23867-bd52-4ded-823a-2ec358ceef9c admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:59.283424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a23867-bd52-4ded-823a-2ec358ceef9c admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:59.288645 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e442aeb-3a0b-44e5-bde1-15a6e4af3d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:59.289432 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e442aeb-3a0b-44e5-bde1-15a6e4af3d2a admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:59.289878 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e442aeb-3a0b-44e5-bde1-15a6e4af3d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:59.290214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e442aeb-3a0b-44e5-bde1-15a6e4af3d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:59.291076 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e442aeb-3a0b-44e5-bde1-15a6e4af3d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:59.294573 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e442aeb-3a0b-44e5-bde1-15a6e4af3d2a admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:59.310701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1349/2705] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:59.315879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2c9e3-5923-4122-a7cd-0c732cab6ab2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:59.354397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2c9e3-5923-4122-a7cd-0c732cab6ab2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:59.371351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7316dd-85cc-4af1-9212-a28df9cd9402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:59.371998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7316dd-85cc-4af1-9212-a28df9cd9402 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:59.372263 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7316dd-85cc-4af1-9212-a28df9cd9402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:59.372530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7316dd-85cc-4af1-9212-a28df9cd9402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:59.373192 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7316dd-85cc-4af1-9212-a28df9cd9402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:59.374904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7316dd-85cc-4af1-9212-a28df9cd9402 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:59.439295 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1357/2706] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:38:59.441899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e442aeb-3a0b-44e5-bde1-15a6e4af3d2a admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:59.454692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e442aeb-3a0b-44e5-bde1-15a6e4af3d2a admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:59.479280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e9183-d307-49a9-8e51-826413a19616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:59.480328 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e9183-d307-49a9-8e51-826413a19616 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:59.480763 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e9183-d307-49a9-8e51-826413a19616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:59.481586 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e9183-d307-49a9-8e51-826413a19616 admin admin] PATH_INFO: `/v3/projects/de71465d137f4bdfbf68116abb043963/users/3fbfb02c32ce470c88698f57d220cc20/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:59.515862 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e9183-d307-49a9-8e51-826413a19616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de71465d137f4bdfbf68116abb043963, user_id=3fbfb02c32ce470c88698f57d220cc20, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:59.518892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e9183-d307-49a9-8e51-826413a19616 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:38:59.547415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1350/2707] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:38:59 2022] PUT /identity/v3/projects/de71465d137f4bdfbf68116abb043963/users/3fbfb02c32ce470c88698f57d220cc20/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:38:59.561048 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd8a90-53b1-4bf9-b41c-99c825ecc4d9 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:59.561397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd8a90-53b1-4bf9-b41c-99c825ecc4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:59.561685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd8a90-53b1-4bf9-b41c-99c825ecc4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:59.624530 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-10bd8a90-53b1-4bf9-b41c-99c825ecc4d9 None None] MFA Rules not processed for user `3fbfb02c32ce470c88698f57d220cc20`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:38:59.633628 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-10bd8a90-53b1-4bf9-b41c-99c825ecc4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:59.764653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1358/2708] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:38:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:38:59.785423 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e9183-d307-49a9-8e51-826413a19616 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:38:59.791905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-761e9183-d307-49a9-8e51-826413a19616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:38:59.798118 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:38:59.960943 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e9183-d307-49a9-8e51-826413a19616 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:38:59.965878 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd422e-05ed-4858-99a9-e0e8485e59f3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a2fb70c9ae4f4b81ac4de513b868cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:38:59.966549 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd422e-05ed-4858-99a9-e0e8485e59f3 service glance] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:38:59.966836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd422e-05ed-4858-99a9-e0e8485e59f3 service glance] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:38:59.967106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd422e-05ed-4858-99a9-e0e8485e59f3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:38:59.979450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd422e-05ed-4858-99a9-e0e8485e59f3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:38:59.980902 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd422e-05ed-4858-99a9-e0e8485e59f3 service glance] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:00.020862 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1351/2709] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:38:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:39:04.857982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26d2c9-a0cc-494f-8111-8780894738ce None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:04.858560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26d2c9-a0cc-494f-8111-8780894738ce None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:04.858560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26d2c9-a0cc-494f-8111-8780894738ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:04.890718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-9c26d2c9-a0cc-494f-8111-8780894738ce None None] MFA Rules not processed for user `5eb42a654af74ea59f0bd614db3c3c0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:04.898525 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9c26d2c9-a0cc-494f-8111-8780894738ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:04.976522 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1359/2710] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:39:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:39:04.989280 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd422e-05ed-4858-99a9-e0e8485e59f3 service glance] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:04.991119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-74cd422e-05ed-4858-99a9-e0e8485e59f3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:05.003682 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd422e-05ed-4858-99a9-e0e8485e59f3 service glance] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:05.009057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205795c9-8e11-4bd8-bfc8-78d7675aa7b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a2fb70c9ae4f4b81ac4de513b868cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:05.009968 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205795c9-8e11-4bd8-bfc8-78d7675aa7b8 service glance] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:05.010816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205795c9-8e11-4bd8-bfc8-78d7675aa7b8 service glance] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:05.011270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205795c9-8e11-4bd8-bfc8-78d7675aa7b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:05.013699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-205795c9-8e11-4bd8-bfc8-78d7675aa7b8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:05.096804 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205795c9-8e11-4bd8-bfc8-78d7675aa7b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:05.098315 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205795c9-8e11-4bd8-bfc8-78d7675aa7b8 service glance] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:05.107320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1352/2711] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:39:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:39:05.257557 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fdc51-dd31-4de6-9a0d-908f8da2ab75 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:05.257858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fdc51-dd31-4de6-9a0d-908f8da2ab75 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:05.258119 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fdc51-dd31-4de6-9a0d-908f8da2ab75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:05.279329 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-512fdc51-dd31-4de6-9a0d-908f8da2ab75 None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:05.280560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-512fdc51-dd31-4de6-9a0d-908f8da2ab75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:05.328640 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:05.347653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1360/2712] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:39:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:39:05.362579 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205795c9-8e11-4bd8-bfc8-78d7675aa7b8 service glance] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:05.364012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-205795c9-8e11-4bd8-bfc8-78d7675aa7b8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:05.444687 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205795c9-8e11-4bd8-bfc8-78d7675aa7b8 service glance] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:05.450612 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbbe20-4705-496b-83d8-acc142264b5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:05.451375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbbe20-4705-496b-83d8-acc142264b5e service swift] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:05.451652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbbe20-4705-496b-83d8-acc142264b5e service swift] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:05.451915 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbbe20-4705-496b-83d8-acc142264b5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:05.461189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbbe20-4705-496b-83d8-acc142264b5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:05.462885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbbe20-4705-496b-83d8-acc142264b5e service swift] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:05.472201 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1353/2713] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:39:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:39:15.202571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb79bd-6e37-4082-806c-04a8d098c30a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:15.202571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb79bd-6e37-4082-806c-04a8d098c30a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:15.202571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb79bd-6e37-4082-806c-04a8d098c30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:15.224525 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-72eb79bd-6e37-4082-806c-04a8d098c30a None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:15.225759 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-72eb79bd-6e37-4082-806c-04a8d098c30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:15.233986 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1361/2714] 10.222.0.23 () {58 vars in 967 bytes} [Fri Sep 9 16:39:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:39:15.248093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbbe20-4705-496b-83d8-acc142264b5e service swift] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:15.260276 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbbe20-4705-496b-83d8-acc142264b5e service swift] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:15.268080 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796a2b0-9069-459f-8482-3cce4c6f8732 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:15.268701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796a2b0-9069-459f-8482-3cce4c6f8732 service swift] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:15.269065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796a2b0-9069-459f-8482-3cce4c6f8732 service swift] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:15.269267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796a2b0-9069-459f-8482-3cce4c6f8732 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:15.283601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796a2b0-9069-459f-8482-3cce4c6f8732 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:15.292669 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796a2b0-9069-459f-8482-3cce4c6f8732 service swift] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:15.306155 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1354/2715] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:39:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:39:15.885081 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335e76e-af58-4d8f-8460-24e095dfe396 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:15.885359 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335e76e-af58-4d8f-8460-24e095dfe396 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:15.885631 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335e76e-af58-4d8f-8460-24e095dfe396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:15.910165 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b335e76e-af58-4d8f-8460-24e095dfe396 None None] MFA Rules not processed for user `f86d1ec821814f628879590513702cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:15.911418 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b335e76e-af58-4d8f-8460-24e095dfe396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:15.920052 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1362/2716] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:39:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:39:16.199081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796a2b0-9069-459f-8482-3cce4c6f8732 service swift] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:16.200774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-b796a2b0-9069-459f-8482-3cce4c6f8732 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:16.234210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:16.273899 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796a2b0-9069-459f-8482-3cce4c6f8732 service swift] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:16.277800 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd73ef-8dae-42bf-a46f-053efb8a58ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:16.278412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd73ef-8dae-42bf-a46f-053efb8a58ad service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:16.278717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd73ef-8dae-42bf-a46f-053efb8a58ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:16.278985 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd73ef-8dae-42bf-a46f-053efb8a58ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:16.280766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-11dd73ef-8dae-42bf-a46f-053efb8a58ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:16.290984 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd73ef-8dae-42bf-a46f-053efb8a58ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:16.292401 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd73ef-8dae-42bf-a46f-053efb8a58ad service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:16.300532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1355/2717] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:39:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:39:19.898476 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191bef2-0222-4edf-a92a-c8324b243c44 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:19.899023 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191bef2-0222-4edf-a92a-c8324b243c44 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:19.899136 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191bef2-0222-4edf-a92a-c8324b243c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:19.918875 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-e191bef2-0222-4edf-a92a-c8324b243c44 None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:19.920107 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-e191bef2-0222-4edf-a92a-c8324b243c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:19.927359 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1363/2718] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Sep 9 16:39:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:39:19.943713 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd73ef-8dae-42bf-a46f-053efb8a58ad service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:19.956735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd73ef-8dae-42bf-a46f-053efb8a58ad service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:19.962376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8060e8-adc6-4292-821c-a837f3bd25cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:19.963256 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8060e8-adc6-4292-821c-a837f3bd25cd service swift] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:19.963802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8060e8-adc6-4292-821c-a837f3bd25cd service swift] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:19.964064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8060e8-adc6-4292-821c-a837f3bd25cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:19.973936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8060e8-adc6-4292-821c-a837f3bd25cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:19.975688 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8060e8-adc6-4292-821c-a837f3bd25cd service swift] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:19.991501 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1356/2719] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:39:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 16:39:26.078238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191bef2-0222-4edf-a92a-c8324b243c44 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:26.175969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191bef2-0222-4edf-a92a-c8324b243c44 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:26.183306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f16d2-94ee-4868-8a45-fa5453f58ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:26.183960 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f16d2-94ee-4868-8a45-fa5453f58ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:26.184327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f16d2-94ee-4868-8a45-fa5453f58ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:26.184603 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f16d2-94ee-4868-8a45-fa5453f58ded admin admin] PATH_INFO: `/v3/users/3fbfb02c32ce470c88698f57d220cc20` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:26.185863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f16d2-94ee-4868-8a45-fa5453f58ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fbfb02c32ce470c88698f57d220cc20)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:26.189998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f16d2-94ee-4868-8a45-fa5453f58ded admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:26.250349 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-510f16d2-94ee-4868-8a45-fa5453f58ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fbfb02c32ce470c88698f57d220cc20'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:26.251065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-510f16d2-94ee-4868-8a45-fa5453f58ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fbfb02c32ce470c88698f57d220cc20'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:26.251787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-510f16d2-94ee-4868-8a45-fa5453f58ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fbfb02c32ce470c88698f57d220cc20'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:26.259093 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-510f16d2-94ee-4868-8a45-fa5453f58ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fbfb02c32ce470c88698f57d220cc20'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:26.270951 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-510f16d2-94ee-4868-8a45-fa5453f58ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fbfb02c32ce470c88698f57d220cc20'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:26.285627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1364/2720] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:39:26 2022] DELETE /identity/v3/users/3fbfb02c32ce470c88698f57d220cc20 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:26.290372 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85f2ab-f461-44d6-bdff-60a4c2e8fa6b None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:26.290950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85f2ab-f461-44d6-bdff-60a4c2e8fa6b None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:26.290950 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85f2ab-f461-44d6-bdff-60a4c2e8fa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:26.313191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-5a85f2ab-f461-44d6-bdff-60a4c2e8fa6b None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:26.314469 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5a85f2ab-f461-44d6-bdff-60a4c2e8fa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:26.381333 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1357/2721] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:39:26.394677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f16d2-94ee-4868-8a45-fa5453f58ded admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:26.396241 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-510f16d2-94ee-4868-8a45-fa5453f58ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:26.457658 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f16d2-94ee-4868-8a45-fa5453f58ded admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:26.461714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77cd12-2a81-469c-8597-714d230f6104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:26.462469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77cd12-2a81-469c-8597-714d230f6104 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:26.462857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77cd12-2a81-469c-8597-714d230f6104 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:26.463196 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77cd12-2a81-469c-8597-714d230f6104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:26.469754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:26.475279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77cd12-2a81-469c-8597-714d230f6104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:26.480593 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77cd12-2a81-469c-8597-714d230f6104 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:26.491892 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1365/2722] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:39:26.798224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85f2ab-f461-44d6-bdff-60a4c2e8fa6b None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:26.800134 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5a85f2ab-f461-44d6-bdff-60a4c2e8fa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:26.825223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85f2ab-f461-44d6-bdff-60a4c2e8fa6b None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:26.830282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba7234-82e7-472e-863f-59abb527e421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:26.831045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba7234-82e7-472e-863f-59abb527e421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:26.831320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba7234-82e7-472e-863f-59abb527e421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:26.831588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba7234-82e7-472e-863f-59abb527e421 admin admin] PATH_INFO: `/v3/projects/de71465d137f4bdfbf68116abb043963` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:26.835073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba7234-82e7-472e-863f-59abb527e421 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de71465d137f4bdfbf68116abb043963)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:26.837185 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba7234-82e7-472e-863f-59abb527e421 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:26.890615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bcba7234-82e7-472e-863f-59abb527e421 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de71465d137f4bdfbf68116abb043963', 'request_id': 'req-bcba7234-82e7-472e-863f-59abb527e421'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:26.897100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bcba7234-82e7-472e-863f-59abb527e421 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de71465d137f4bdfbf68116abb043963', 'request_id': 'req-bcba7234-82e7-472e-863f-59abb527e421'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:26.906450 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bcba7234-82e7-472e-863f-59abb527e421 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de71465d137f4bdfbf68116abb043963', 'request_id': 'req-bcba7234-82e7-472e-863f-59abb527e421'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:26.919567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bcba7234-82e7-472e-863f-59abb527e421 admin admin] The token cache is being invalidate because project de71465d137f4bdfbf68116abb043963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:39:26.919844 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bcba7234-82e7-472e-863f-59abb527e421 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:26.920486 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-bcba7234-82e7-472e-863f-59abb527e421 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:26.922233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1358/2723] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:39:26 2022] DELETE /identity/v3/projects/de71465d137f4bdfbf68116abb043963 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:26.928736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d56d1-60d0-4037-bef0-e697ac99b04e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:26.929026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d56d1-60d0-4037-bef0-e697ac99b04e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:26.929336 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d56d1-60d0-4037-bef0-e697ac99b04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:26.952959 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c27d56d1-60d0-4037-bef0-e697ac99b04e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:26.954330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c27d56d1-60d0-4037-bef0-e697ac99b04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:27.025722 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1366/2724] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:39:27.100155 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f91811-9d8b-4f86-be18-d912cc4edbac None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:27.100411 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f91811-9d8b-4f86-be18-d912cc4edbac None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:27.100667 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f91811-9d8b-4f86-be18-d912cc4edbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:27.126366 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-e1f91811-9d8b-4f86-be18-d912cc4edbac None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:27.127626 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e1f91811-9d8b-4f86-be18-d912cc4edbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:27.139472 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1359/2725] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:39:27.140304 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d56d1-60d0-4037-bef0-e697ac99b04e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:27.152251 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d56d1-60d0-4037-bef0-e697ac99b04e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:27.160354 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c632b2-a86f-4b6b-9a0d-fe69efbaa4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:27.161100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c632b2-a86f-4b6b-9a0d-fe69efbaa4ef admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:27.161474 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c632b2-a86f-4b6b-9a0d-fe69efbaa4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:27.162207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c632b2-a86f-4b6b-9a0d-fe69efbaa4ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:27.163090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c632b2-a86f-4b6b-9a0d-fe69efbaa4ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:39:27.163667 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c632b2-a86f-4b6b-9a0d-fe69efbaa4ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:27.165743 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c632b2-a86f-4b6b-9a0d-fe69efbaa4ef admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:27.194337 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1367/2726] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 16:39:27.195959 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f91811-9d8b-4f86-be18-d912cc4edbac None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:27.205758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f91811-9d8b-4f86-be18-d912cc4edbac None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:27.211188 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cb559-613a-4eaf-8cee-4e2f26368688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:27.211889 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb559-613a-4eaf-8cee-4e2f26368688 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:27.212233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb559-613a-4eaf-8cee-4e2f26368688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:27.212625 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb559-613a-4eaf-8cee-4e2f26368688 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:27.213946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2cb559-613a-4eaf-8cee-4e2f26368688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:27.216108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2cb559-613a-4eaf-8cee-4e2f26368688 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:27.282556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1360/2727] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:39:27 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:39:27.285689 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c632b2-a86f-4b6b-9a0d-fe69efbaa4ef admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:27.298401 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c632b2-a86f-4b6b-9a0d-fe69efbaa4ef admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:27.382487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc57243-8fb9-4ab9-9d2f-8e6e0e92c189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:27.383246 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc57243-8fb9-4ab9-9d2f-8e6e0e92c189 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:27.383590 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc57243-8fb9-4ab9-9d2f-8e6e0e92c189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:27.384039 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc57243-8fb9-4ab9-9d2f-8e6e0e92c189 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:27.384631 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc57243-8fb9-4ab9-9d2f-8e6e0e92c189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:27.386843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc57243-8fb9-4ab9-9d2f-8e6e0e92c189 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:27.412837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1368/2728] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:39:27 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:39:27.416850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cb559-613a-4eaf-8cee-4e2f26368688 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:27.429235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:27.432766 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cb559-613a-4eaf-8cee-4e2f26368688 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:27.438801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0f80e-2ea1-4a67-b1b2-fbe630258862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:27.439897 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0f80e-2ea1-4a67-b1b2-fbe630258862 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:27.440416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0f80e-2ea1-4a67-b1b2-fbe630258862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:27.440673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0f80e-2ea1-4a67-b1b2-fbe630258862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:27.441339 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0f80e-2ea1-4a67-b1b2-fbe630258862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:27.443074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0f80e-2ea1-4a67-b1b2-fbe630258862 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:27.459424 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1361/2729] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:39:27.464523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc57243-8fb9-4ab9-9d2f-8e6e0e92c189 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:27.478140 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc57243-8fb9-4ab9-9d2f-8e6e0e92c189 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:27.493857 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7ab27-a49d-4412-90b9-4f81c10b0b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:27.494617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7ab27-a49d-4412-90b9-4f81c10b0b1e admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:27.494936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7ab27-a49d-4412-90b9-4f81c10b0b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:27.495204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7ab27-a49d-4412-90b9-4f81c10b0b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:27.495954 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc7ab27-a49d-4412-90b9-4f81c10b0b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:27.498100 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc7ab27-a49d-4412-90b9-4f81c10b0b1e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:27.515154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1369/2730] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:39:27.519479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0f80e-2ea1-4a67-b1b2-fbe630258862 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:27.535397 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0f80e-2ea1-4a67-b1b2-fbe630258862 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:27.541755 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5db50b-2b0b-46a0-b264-cbc88962319e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:27.542553 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5db50b-2b0b-46a0-b264-cbc88962319e admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:27.542892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5db50b-2b0b-46a0-b264-cbc88962319e admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:27.543382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5db50b-2b0b-46a0-b264-cbc88962319e admin admin] PATH_INFO: `/v3/projects/03261f6dc75e4e3c9c0b0575fe75f755/users/da818e1c2d284477a290c4a4ec80c451/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:27.563249 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5db50b-2b0b-46a0-b264-cbc88962319e admin admin] RBAC: Authorizing `identity:create_grant(project_id=03261f6dc75e4e3c9c0b0575fe75f755, user_id=da818e1c2d284477a290c4a4ec80c451, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:27.567170 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5db50b-2b0b-46a0-b264-cbc88962319e admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:27.598154 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1362/2731] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:39:27 2022] PUT /identity/v3/projects/03261f6dc75e4e3c9c0b0575fe75f755/users/da818e1c2d284477a290c4a4ec80c451/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:27.599969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0615a0-f681-43b6-861c-b472ebbe9882 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:27.600258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0615a0-f681-43b6-861c-b472ebbe9882 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:27.600535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0615a0-f681-43b6-861c-b472ebbe9882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:27.631874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-ef0615a0-f681-43b6-861c-b472ebbe9882 None None] MFA Rules not processed for user `da818e1c2d284477a290c4a4ec80c451`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:27.633120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ef0615a0-f681-43b6-861c-b472ebbe9882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:27.690028 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:27.710491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1370/2732] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:39:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:39:27.715204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5db50b-2b0b-46a0-b264-cbc88962319e admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:27.784820 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5db50b-2b0b-46a0-b264-cbc88962319e admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:27.789908 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f71d00-a317-4f7f-ac6f-11dda0168011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:27.790601 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f71d00-a317-4f7f-ac6f-11dda0168011 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:27.790939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f71d00-a317-4f7f-ac6f-11dda0168011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:27.791269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f71d00-a317-4f7f-ac6f-11dda0168011 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:27.791954 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f71d00-a317-4f7f-ac6f-11dda0168011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:27.794249 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f71d00-a317-4f7f-ac6f-11dda0168011 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:27.831439 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1363/2733] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:39:27 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:39:27.832679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0615a0-f681-43b6-861c-b472ebbe9882 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:27.843842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0615a0-f681-43b6-861c-b472ebbe9882 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:27.908213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbf344-6725-4012-b67a-0dfbe2156e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:27.908923 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbf344-6725-4012-b67a-0dfbe2156e09 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:27.909262 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbf344-6725-4012-b67a-0dfbe2156e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:27.909659 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbf344-6725-4012-b67a-0dfbe2156e09 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:27.911274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbf344-6725-4012-b67a-0dfbe2156e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:27.913719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbf344-6725-4012-b67a-0dfbe2156e09 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:27.940956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1371/2734] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:39:27 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:39:27.944952 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f71d00-a317-4f7f-ac6f-11dda0168011 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:27.955572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f71d00-a317-4f7f-ac6f-11dda0168011 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:27.960522 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de7ba3-f33a-4790-8626-9522c5e81f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:27.961086 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de7ba3-f33a-4790-8626-9522c5e81f95 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:27.961216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de7ba3-f33a-4790-8626-9522c5e81f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:27.961485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de7ba3-f33a-4790-8626-9522c5e81f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:27.962413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de7ba3-f33a-4790-8626-9522c5e81f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:27.963827 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de7ba3-f33a-4790-8626-9522c5e81f95 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:27.975019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1364/2735] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:39:27.979346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbf344-6725-4012-b67a-0dfbe2156e09 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:27.993489 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbf344-6725-4012-b67a-0dfbe2156e09 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:27.998435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afea353-8042-4b7e-bb16-8efbe740f510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:27.999090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afea353-8042-4b7e-bb16-8efbe740f510 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:27.999369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afea353-8042-4b7e-bb16-8efbe740f510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:27.999643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afea353-8042-4b7e-bb16-8efbe740f510 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:28.000309 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afea353-8042-4b7e-bb16-8efbe740f510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:28.001930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afea353-8042-4b7e-bb16-8efbe740f510 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:28.016957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1372/2736] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:39:28.018439 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de7ba3-f33a-4790-8626-9522c5e81f95 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:28.029307 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de7ba3-f33a-4790-8626-9522c5e81f95 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:28.034263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee096bac-22a4-436a-b61a-e56ca18511af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:28.035194 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee096bac-22a4-436a-b61a-e56ca18511af admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:28.035485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee096bac-22a4-436a-b61a-e56ca18511af admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:28.035844 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee096bac-22a4-436a-b61a-e56ca18511af admin admin] PATH_INFO: `/v3/projects/bb7f1009c353487cb79a867021e092ae/users/b29e28533a91456b859da7374f5852b5/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:28.052192 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee096bac-22a4-436a-b61a-e56ca18511af admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb7f1009c353487cb79a867021e092ae, user_id=b29e28533a91456b859da7374f5852b5, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:28.055056 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee096bac-22a4-436a-b61a-e56ca18511af admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:28.070228 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1365/2737] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:39:28 2022] PUT /identity/v3/projects/bb7f1009c353487cb79a867021e092ae/users/b29e28533a91456b859da7374f5852b5/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:28.084653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc4ba6-28ca-455d-9130-7df91f63e718 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:28.084931 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc4ba6-28ca-455d-9130-7df91f63e718 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:28.086633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc4ba6-28ca-455d-9130-7df91f63e718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:28.117397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-68cc4ba6-28ca-455d-9130-7df91f63e718 None None] MFA Rules not processed for user `b29e28533a91456b859da7374f5852b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:28.118689 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-68cc4ba6-28ca-455d-9130-7df91f63e718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:28.187661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1373/2738] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:39:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:39:28.192273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee096bac-22a4-436a-b61a-e56ca18511af admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:28.257441 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee096bac-22a4-436a-b61a-e56ca18511af admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:28.263090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166ee80-5993-4b91-8ff6-8db4b611efb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:28.263995 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166ee80-5993-4b91-8ff6-8db4b611efb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:28.264344 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166ee80-5993-4b91-8ff6-8db4b611efb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:28.264735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166ee80-5993-4b91-8ff6-8db4b611efb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:28.265523 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166ee80-5993-4b91-8ff6-8db4b611efb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:28.268048 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166ee80-5993-4b91-8ff6-8db4b611efb6 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:28.319444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1366/2739] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:39:28 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:39:28.323828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc4ba6-28ca-455d-9130-7df91f63e718 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:28.335508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc4ba6-28ca-455d-9130-7df91f63e718 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:28.400026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:28.406290 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf67f91-ca81-4be4-afa8-fe559e655d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:28.407267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf67f91-ca81-4be4-afa8-fe559e655d33 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:28.407649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf67f91-ca81-4be4-afa8-fe559e655d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:28.408442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf67f91-ca81-4be4-afa8-fe559e655d33 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:28.409451 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf67f91-ca81-4be4-afa8-fe559e655d33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:28.412143 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf67f91-ca81-4be4-afa8-fe559e655d33 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:28.442238 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1374/2740] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:39:28 2022] POST /identity/v3/users => generated 298 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:39:28.446373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166ee80-5993-4b91-8ff6-8db4b611efb6 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:28.453495 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:28.459108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166ee80-5993-4b91-8ff6-8db4b611efb6 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:28.468867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c1393-8038-405a-babe-5055e8f32d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:28.469492 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c1393-8038-405a-babe-5055e8f32d13 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:28.469772 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c1393-8038-405a-babe-5055e8f32d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:28.470029 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c1393-8038-405a-babe-5055e8f32d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:28.470714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c1393-8038-405a-babe-5055e8f32d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:28.472620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c1393-8038-405a-babe-5055e8f32d13 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:28.484853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1367/2741] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:39:28.489191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf67f91-ca81-4be4-afa8-fe559e655d33 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:28.499547 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf67f91-ca81-4be4-afa8-fe559e655d33 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:28.504244 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3c4d7-95f4-44b5-8f57-f21e2c7ead29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:28.504995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3c4d7-95f4-44b5-8f57-f21e2c7ead29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:28.505325 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3c4d7-95f4-44b5-8f57-f21e2c7ead29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:28.505699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3c4d7-95f4-44b5-8f57-f21e2c7ead29 admin admin] PATH_INFO: `/v3/projects/bde372aab3484144bdfa06b13fd515a1/users/fbc73c0381e24d69abccf790039d5352/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:28.526191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3c4d7-95f4-44b5-8f57-f21e2c7ead29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde372aab3484144bdfa06b13fd515a1, user_id=fbc73c0381e24d69abccf790039d5352, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:28.529868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3c4d7-95f4-44b5-8f57-f21e2c7ead29 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:28.545861 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1375/2742] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:39:28 2022] PUT /identity/v3/projects/bde372aab3484144bdfa06b13fd515a1/users/fbc73c0381e24d69abccf790039d5352/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:28.550780 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c1393-8038-405a-babe-5055e8f32d13 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:28.564033 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c1393-8038-405a-babe-5055e8f32d13 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:28.632721 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe9c6c-ed2d-4681-a0ca-dd2a8f58de84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:28.632964 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe9c6c-ed2d-4681-a0ca-dd2a8f58de84 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:28.633341 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe9c6c-ed2d-4681-a0ca-dd2a8f58de84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:28.633712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe9c6c-ed2d-4681-a0ca-dd2a8f58de84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:28.634460 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe9c6c-ed2d-4681-a0ca-dd2a8f58de84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:28.636270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe9c6c-ed2d-4681-a0ca-dd2a8f58de84 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:28.647829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1368/2743] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:39:28.654406 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3c4d7-95f4-44b5-8f57-f21e2c7ead29 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:28.665314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3c4d7-95f4-44b5-8f57-f21e2c7ead29 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:28.669919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bef532-115a-4155-8351-ed94dd8a79fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:28.670602 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bef532-115a-4155-8351-ed94dd8a79fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:28.670879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bef532-115a-4155-8351-ed94dd8a79fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:28.671149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bef532-115a-4155-8351-ed94dd8a79fc admin admin] PATH_INFO: `/v3/projects/bde372aab3484144bdfa06b13fd515a1/users/fbc73c0381e24d69abccf790039d5352/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:28.673063 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bef532-115a-4155-8351-ed94dd8a79fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde372aab3484144bdfa06b13fd515a1, user_id=fbc73c0381e24d69abccf790039d5352, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:28.675756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bef532-115a-4155-8351-ed94dd8a79fc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:28.694714 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1376/2744] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:39:28 2022] PUT /identity/v3/projects/bde372aab3484144bdfa06b13fd515a1/users/fbc73c0381e24d69abccf790039d5352/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:28.705596 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f692c-3fb4-481a-8922-20d8e4ae41b4 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:28.705886 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f692c-3fb4-481a-8922-20d8e4ae41b4 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:28.706239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f692c-3fb4-481a-8922-20d8e4ae41b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:28.737236 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-d74f692c-3fb4-481a-8922-20d8e4ae41b4 None None] MFA Rules not processed for user `fbc73c0381e24d69abccf790039d5352`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:28.738475 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d74f692c-3fb4-481a-8922-20d8e4ae41b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:28.816434 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1369/2745] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:39:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:39:28.829192 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bef532-115a-4155-8351-ed94dd8a79fc admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:28.830989 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-56bef532-115a-4155-8351-ed94dd8a79fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:28.893361 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bef532-115a-4155-8351-ed94dd8a79fc admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:28.897128 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c2da9-b24b-437b-b278-932c48ca0961 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:28.897741 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c2da9-b24b-437b-b278-932c48ca0961 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:28.898013 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c2da9-b24b-437b-b278-932c48ca0961 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:28.898307 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c2da9-b24b-437b-b278-932c48ca0961 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:28.958067 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c2da9-b24b-437b-b278-932c48ca0961 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:28.959585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c2da9-b24b-437b-b278-932c48ca0961 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:28.969242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1377/2746] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:39:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:39:30.520759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f692c-3fb4-481a-8922-20d8e4ae41b4 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:30.531545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f692c-3fb4-481a-8922-20d8e4ae41b4 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:30.535446 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daef8b-f700-4e4b-b907-fc27beb3e98e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:30.536208 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daef8b-f700-4e4b-b907-fc27beb3e98e service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:30.536446 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daef8b-f700-4e4b-b907-fc27beb3e98e service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:30.537106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daef8b-f700-4e4b-b907-fc27beb3e98e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:30.615714 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66daef8b-f700-4e4b-b907-fc27beb3e98e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:30.617216 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66daef8b-f700-4e4b-b907-fc27beb3e98e service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:30.626939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1370/2747] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:39:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:39:31.282092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c2da9-b24b-437b-b278-932c48ca0961 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:31.292103 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c2da9-b24b-437b-b278-932c48ca0961 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:31.296747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09e6b6-31e3-475b-babe-256dd92cf968 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:31.297468 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e6b6-31e3-475b-babe-256dd92cf968 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:31.297845 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e6b6-31e3-475b-babe-256dd92cf968 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:31.298259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e6b6-31e3-475b-babe-256dd92cf968 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:31.308303 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09e6b6-31e3-475b-babe-256dd92cf968 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:31.309848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09e6b6-31e3-475b-babe-256dd92cf968 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:31.320756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1378/2748] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:39:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:39:35.244269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daef8b-f700-4e4b-b907-fc27beb3e98e service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:35.258131 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=100 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:35.312917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daef8b-f700-4e4b-b907-fc27beb3e98e service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:35.317855 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965174c3-2e6c-4c5e-85f2-4a1f56e924d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:35.318490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965174c3-2e6c-4c5e-85f2-4a1f56e924d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:35.318818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965174c3-2e6c-4c5e-85f2-4a1f56e924d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:35.319089 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965174c3-2e6c-4c5e-85f2-4a1f56e924d4 admin admin] PATH_INFO: `/v3/users/da818e1c2d284477a290c4a4ec80c451` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:35.320248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965174c3-2e6c-4c5e-85f2-4a1f56e924d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da818e1c2d284477a290c4a4ec80c451)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:35.322163 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965174c3-2e6c-4c5e-85f2-4a1f56e924d4 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:35.373219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-965174c3-2e6c-4c5e-85f2-4a1f56e924d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da818e1c2d284477a290c4a4ec80c451'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:35.373927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-965174c3-2e6c-4c5e-85f2-4a1f56e924d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da818e1c2d284477a290c4a4ec80c451'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:35.374547 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-965174c3-2e6c-4c5e-85f2-4a1f56e924d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da818e1c2d284477a290c4a4ec80c451'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:35.382027 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-965174c3-2e6c-4c5e-85f2-4a1f56e924d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da818e1c2d284477a290c4a4ec80c451'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:35.394879 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-965174c3-2e6c-4c5e-85f2-4a1f56e924d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da818e1c2d284477a290c4a4ec80c451'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:35.411979 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1371/2749] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:39:35 2022] DELETE /identity/v3/users/da818e1c2d284477a290c4a4ec80c451 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:35.416591 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683c53b-e316-4fd4-abd7-099c61e9e730 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:35.417084 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683c53b-e316-4fd4-abd7-099c61e9e730 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:35.417237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683c53b-e316-4fd4-abd7-099c61e9e730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:35.439207 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-0683c53b-e316-4fd4-abd7-099c61e9e730 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:35.440430 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0683c53b-e316-4fd4-abd7-099c61e9e730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:35.508752 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:35.526967 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1379/2750] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:39:35.538538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965174c3-2e6c-4c5e-85f2-4a1f56e924d4 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:35.540059 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-965174c3-2e6c-4c5e-85f2-4a1f56e924d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:35.599861 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965174c3-2e6c-4c5e-85f2-4a1f56e924d4 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:35.603590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1ae35-7f16-4f64-8909-bd1c0db1f480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:35.604203 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1ae35-7f16-4f64-8909-bd1c0db1f480 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:35.604479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1ae35-7f16-4f64-8909-bd1c0db1f480 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:35.604744 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1ae35-7f16-4f64-8909-bd1c0db1f480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:35.614258 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1ae35-7f16-4f64-8909-bd1c0db1f480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:35.615810 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1ae35-7f16-4f64-8909-bd1c0db1f480 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:35.625945 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1372/2751] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:39:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:39:36.021428 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683c53b-e316-4fd4-abd7-099c61e9e730 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:36.023565 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0683c53b-e316-4fd4-abd7-099c61e9e730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:36.036094 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683c53b-e316-4fd4-abd7-099c61e9e730 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:36.041214 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:36.041833 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:36.042107 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:36.042376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] PATH_INFO: `/v3/projects/03261f6dc75e4e3c9c0b0575fe75f755` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:36.043514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] RBAC: Authorizing `identity:delete_project(project_id=03261f6dc75e4e3c9c0b0575fe75f755)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:36.045495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:36.105752 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03261f6dc75e4e3c9c0b0575fe75f755', 'request_id': 'req-77e43252-7e30-4aa7-bcb9-a0de7903e61d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.112651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03261f6dc75e4e3c9c0b0575fe75f755', 'request_id': 'req-77e43252-7e30-4aa7-bcb9-a0de7903e61d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.118987 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03261f6dc75e4e3c9c0b0575fe75f755', 'request_id': 'req-77e43252-7e30-4aa7-bcb9-a0de7903e61d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.135709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] The token cache is being invalidate because project 03261f6dc75e4e3c9c0b0575fe75f755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:39:36.135983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.136716 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.142585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1380/2752] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:39:36 2022] DELETE /identity/v3/projects/03261f6dc75e4e3c9c0b0575fe75f755 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:36.142771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1ae35-7f16-4f64-8909-bd1c0db1f480 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:36.153307 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1ae35-7f16-4f64-8909-bd1c0db1f480 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:36.191365 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:36.224014 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392dd6d-c51a-4961-8050-ff6649a4a352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:36.225158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392dd6d-c51a-4961-8050-ff6649a4a352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:36.226355 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392dd6d-c51a-4961-8050-ff6649a4a352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:36.226913 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392dd6d-c51a-4961-8050-ff6649a4a352 admin admin] PATH_INFO: `/v3/users/b29e28533a91456b859da7374f5852b5` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:36.228359 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392dd6d-c51a-4961-8050-ff6649a4a352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b29e28533a91456b859da7374f5852b5)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:36.230612 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392dd6d-c51a-4961-8050-ff6649a4a352 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:36.282940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2392dd6d-c51a-4961-8050-ff6649a4a352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b29e28533a91456b859da7374f5852b5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.283611 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2392dd6d-c51a-4961-8050-ff6649a4a352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b29e28533a91456b859da7374f5852b5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.284189 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2392dd6d-c51a-4961-8050-ff6649a4a352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b29e28533a91456b859da7374f5852b5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.291824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2392dd6d-c51a-4961-8050-ff6649a4a352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b29e28533a91456b859da7374f5852b5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.303545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2392dd6d-c51a-4961-8050-ff6649a4a352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b29e28533a91456b859da7374f5852b5'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.319520 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1373/2753] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:39:36 2022] DELETE /identity/v3/users/b29e28533a91456b859da7374f5852b5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:36.611191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:36.614339 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:36.635712 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e43252-7e30-4aa7-bcb9-a0de7903e61d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:36.701003 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:36.701690 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:36.701969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:36.702240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] PATH_INFO: `/v3/projects/bb7f1009c353487cb79a867021e092ae` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:36.703427 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb7f1009c353487cb79a867021e092ae)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:36.705351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:36.859234 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb7f1009c353487cb79a867021e092ae', 'request_id': 'req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.866665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb7f1009c353487cb79a867021e092ae', 'request_id': 'req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.876711 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb7f1009c353487cb79a867021e092ae', 'request_id': 'req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.892862 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] The token cache is being invalidate because project bb7f1009c353487cb79a867021e092ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:39:36.893165 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.893919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:36.895795 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1381/2754] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:39:36 2022] DELETE /identity/v3/projects/bb7f1009c353487cb79a867021e092ae => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:36.900102 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392dd6d-c51a-4961-8050-ff6649a4a352 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:36.901894 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2392dd6d-c51a-4961-8050-ff6649a4a352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:36.917331 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392dd6d-c51a-4961-8050-ff6649a4a352 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:36.996686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a6340-3d15-47cf-bd3b-aa43c2e11801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:36.997532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a6340-3d15-47cf-bd3b-aa43c2e11801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:36.998461 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a6340-3d15-47cf-bd3b-aa43c2e11801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:36.998932 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a6340-3d15-47cf-bd3b-aa43c2e11801 admin admin] PATH_INFO: `/v3/users/fbc73c0381e24d69abccf790039d5352` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:37.000319 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a6340-3d15-47cf-bd3b-aa43c2e11801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbc73c0381e24d69abccf790039d5352)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:37.002266 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a6340-3d15-47cf-bd3b-aa43c2e11801 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:37.060488 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ec3a6340-3d15-47cf-bd3b-aa43c2e11801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbc73c0381e24d69abccf790039d5352'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:37.061223 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ec3a6340-3d15-47cf-bd3b-aa43c2e11801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbc73c0381e24d69abccf790039d5352'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:37.061913 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ec3a6340-3d15-47cf-bd3b-aa43c2e11801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbc73c0381e24d69abccf790039d5352'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:37.069097 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ec3a6340-3d15-47cf-bd3b-aa43c2e11801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbc73c0381e24d69abccf790039d5352'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:37.070258 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:37.084934 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ec3a6340-3d15-47cf-bd3b-aa43c2e11801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbc73c0381e24d69abccf790039d5352'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:37.105057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1374/2755] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:39:36 2022] DELETE /identity/v3/users/fbc73c0381e24d69abccf790039d5352 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:37.395423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:37.396965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:37.407317 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:37.412261 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc71a2c-2ec4-49af-8b91-c5161aa10e7b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:37.472533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:37.473171 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:37.473451 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:37.473715 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14 admin admin] PATH_INFO: `/v3/projects/bde372aab3484144bdfa06b13fd515a1` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:37.474932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde372aab3484144bdfa06b13fd515a1)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:37.476848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:37.531787 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde372aab3484144bdfa06b13fd515a1', 'request_id': 'req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:37.539264 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde372aab3484144bdfa06b13fd515a1', 'request_id': 'req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:37.546600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde372aab3484144bdfa06b13fd515a1', 'request_id': 'req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:37.562080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14 admin admin] The token cache is being invalidate because project bde372aab3484144bdfa06b13fd515a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:39:37.562532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:37.563252 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-53183d4a-50b1-460d-aa4d-d9f1a6e63f14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:37.565697 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1382/2756] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:39:37 2022] DELETE /identity/v3/projects/bde372aab3484144bdfa06b13fd515a1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:37.571194 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cf320-8c74-4886-a917-6421169e0d3e None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:37.571725 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cf320-8c74-4886-a917-6421169e0d3e None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:37.571725 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cf320-8c74-4886-a917-6421169e0d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:37.592881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-e06cf320-8c74-4886-a917-6421169e0d3e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:37.594664 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e06cf320-8c74-4886-a917-6421169e0d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:37.663295 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1375/2757] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:39:37.671654 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7204c-ca6d-4e57-a37c-b9078eff4df6 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:37.672149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7204c-ca6d-4e57-a37c-b9078eff4df6 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:37.672251 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7204c-ca6d-4e57-a37c-b9078eff4df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:37.693935 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-cde7204c-ca6d-4e57-a37c-b9078eff4df6 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:37.695170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-cde7204c-ca6d-4e57-a37c-b9078eff4df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:37.704938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1383/2758] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:39:37.710093 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cf320-8c74-4886-a917-6421169e0d3e None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:37.725027 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cf320-8c74-4886-a917-6421169e0d3e None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:37.733411 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cd050-443a-4b4c-9f40-dba29c9bd8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:37.734010 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cd050-443a-4b4c-9f40-dba29c9bd8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:37.734276 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cd050-443a-4b4c-9f40-dba29c9bd8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:37.734580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cd050-443a-4b4c-9f40-dba29c9bd8b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:37.735224 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cd050-443a-4b4c-9f40-dba29c9bd8b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:39:37.735518 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cd050-443a-4b4c-9f40-dba29c9bd8b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:37.737107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cd050-443a-4b4c-9f40-dba29c9bd8b0 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:37.749715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1376/2759] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:39:37.754322 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7204c-ca6d-4e57-a37c-b9078eff4df6 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:37.764014 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7204c-ca6d-4e57-a37c-b9078eff4df6 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:37.769828 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b70a7-4704-4a5c-bbfb-50d0d255abc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:37.770435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b70a7-4704-4a5c-bbfb-50d0d255abc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:37.770730 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b70a7-4704-4a5c-bbfb-50d0d255abc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:37.770991 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b70a7-4704-4a5c-bbfb-50d0d255abc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:37.771628 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b70a7-4704-4a5c-bbfb-50d0d255abc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:37.773535 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b70a7-4704-4a5c-bbfb-50d0d255abc0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:37.820726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1384/2760] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:39:37 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:39:37.824888 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cd050-443a-4b4c-9f40-dba29c9bd8b0 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:37.835299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cd050-443a-4b4c-9f40-dba29c9bd8b0 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:37.893507 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4daf6e-8231-41ef-8146-bc3fc1461b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:37.894213 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4daf6e-8231-41ef-8146-bc3fc1461b3d admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:37.894591 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4daf6e-8231-41ef-8146-bc3fc1461b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:37.894918 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4daf6e-8231-41ef-8146-bc3fc1461b3d admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:37.895681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4daf6e-8231-41ef-8146-bc3fc1461b3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:37.897853 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4daf6e-8231-41ef-8146-bc3fc1461b3d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:37.919517 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1377/2761] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:39:37 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:39:37.924161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b70a7-4704-4a5c-bbfb-50d0d255abc0 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:37.936735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b70a7-4704-4a5c-bbfb-50d0d255abc0 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:37.941814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54de19f-d814-4f9c-b2b1-249c5c0aac7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:37.942718 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54de19f-d814-4f9c-b2b1-249c5c0aac7a admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:37.942943 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54de19f-d814-4f9c-b2b1-249c5c0aac7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:37.943210 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54de19f-d814-4f9c-b2b1-249c5c0aac7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:37.943851 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54de19f-d814-4f9c-b2b1-249c5c0aac7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:37.946053 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54de19f-d814-4f9c-b2b1-249c5c0aac7a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:37.960180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1385/2762] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:39:37.965465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4daf6e-8231-41ef-8146-bc3fc1461b3d admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:37.977787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4daf6e-8231-41ef-8146-bc3fc1461b3d admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:37.984867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695faac-5e52-42be-882f-ec35bc3c2f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:37.984867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695faac-5e52-42be-882f-ec35bc3c2f62 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:37.984867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695faac-5e52-42be-882f-ec35bc3c2f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:37.985166 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695faac-5e52-42be-882f-ec35bc3c2f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:37.985549 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695faac-5e52-42be-882f-ec35bc3c2f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:37.988045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695faac-5e52-42be-882f-ec35bc3c2f62 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:37.999929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1378/2763] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:39:38.004424 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54de19f-d814-4f9c-b2b1-249c5c0aac7a admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:38.015313 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54de19f-d814-4f9c-b2b1-249c5c0aac7a admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:38.020242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbea242-afb4-4310-909d-48e4d218e912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:38.020984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbea242-afb4-4310-909d-48e4d218e912 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:38.021250 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbea242-afb4-4310-909d-48e4d218e912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:38.021525 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbea242-afb4-4310-909d-48e4d218e912 admin admin] PATH_INFO: `/v3/projects/d45e5a66699e4a8fbe4322b09360f11c/users/2bee53776d244e0588262571d762eee9/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:38.037237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbea242-afb4-4310-909d-48e4d218e912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d45e5a66699e4a8fbe4322b09360f11c, user_id=2bee53776d244e0588262571d762eee9, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:38.039956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbea242-afb4-4310-909d-48e4d218e912 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:38.055593 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1386/2764] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:39:38 2022] PUT /identity/v3/projects/d45e5a66699e4a8fbe4322b09360f11c/users/2bee53776d244e0588262571d762eee9/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:38.067358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3b8c-acf2-41a9-bd6d-d3cc3d493375 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:38.067653 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3b8c-acf2-41a9-bd6d-d3cc3d493375 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:38.067924 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3b8c-acf2-41a9-bd6d-d3cc3d493375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:38.096974 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-e2bf3b8c-acf2-41a9-bd6d-d3cc3d493375 None None] MFA Rules not processed for user `2bee53776d244e0588262571d762eee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:38.098243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e2bf3b8c-acf2-41a9-bd6d-d3cc3d493375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:38.128554 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:38.165621 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1379/2765] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:39:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:39:38.177454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbea242-afb4-4310-909d-48e4d218e912 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:38.179239 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-2dbea242-afb4-4310-909d-48e4d218e912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:38.212202 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:38.236581 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbea242-afb4-4310-909d-48e4d218e912 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:38.240376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fce3c-ce08-4c29-9538-32f88d3382f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:38.240972 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fce3c-ce08-4c29-9538-32f88d3382f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:38.241244 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fce3c-ce08-4c29-9538-32f88d3382f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:38.241507 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fce3c-ce08-4c29-9538-32f88d3382f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:38.250352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fce3c-ce08-4c29-9538-32f88d3382f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:38.251815 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fce3c-ce08-4c29-9538-32f88d3382f8 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:38.260641 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1387/2766] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:39:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:39:38.286808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf3b8c-acf2-41a9-bd6d-d3cc3d493375 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:38.349019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf3b8c-acf2-41a9-bd6d-d3cc3d493375 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:38.353841 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcaa94-32a4-476b-b06c-18fb1962dbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:38.354466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcaa94-32a4-476b-b06c-18fb1962dbda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:38.354779 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcaa94-32a4-476b-b06c-18fb1962dbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:38.355044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcaa94-32a4-476b-b06c-18fb1962dbda admin admin] PATH_INFO: `/v3/users/2bee53776d244e0588262571d762eee9` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:38.356178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcaa94-32a4-476b-b06c-18fb1962dbda admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bee53776d244e0588262571d762eee9)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:38.358092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcaa94-32a4-476b-b06c-18fb1962dbda admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:38.406889 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-18dcaa94-32a4-476b-b06c-18fb1962dbda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bee53776d244e0588262571d762eee9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:38.407567 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-18dcaa94-32a4-476b-b06c-18fb1962dbda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bee53776d244e0588262571d762eee9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:38.408204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-18dcaa94-32a4-476b-b06c-18fb1962dbda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bee53776d244e0588262571d762eee9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:38.415590 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-18dcaa94-32a4-476b-b06c-18fb1962dbda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bee53776d244e0588262571d762eee9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:38.427953 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-18dcaa94-32a4-476b-b06c-18fb1962dbda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bee53776d244e0588262571d762eee9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:38.443062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1380/2767] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:39:38 2022] DELETE /identity/v3/users/2bee53776d244e0588262571d762eee9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:38.447820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a22a07-24bb-47e1-ae79-49eb614ed8fc None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:38.448102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a22a07-24bb-47e1-ae79-49eb614ed8fc None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:38.448374 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a22a07-24bb-47e1-ae79-49eb614ed8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:38.467654 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-92a22a07-24bb-47e1-ae79-49eb614ed8fc None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:38.468849 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-92a22a07-24bb-47e1-ae79-49eb614ed8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:38.533994 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1388/2768] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:39:38.544178 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcaa94-32a4-476b-b06c-18fb1962dbda admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:38.545681 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-18dcaa94-32a4-476b-b06c-18fb1962dbda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:38.601383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:38.605658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcaa94-32a4-476b-b06c-18fb1962dbda admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:38.611177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea4409-dc8f-4994-a100-2cbfaee5b853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:38.611844 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea4409-dc8f-4994-a100-2cbfaee5b853 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:38.612122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea4409-dc8f-4994-a100-2cbfaee5b853 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:38.612402 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea4409-dc8f-4994-a100-2cbfaee5b853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:38.622464 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea4409-dc8f-4994-a100-2cbfaee5b853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:38.623956 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea4409-dc8f-4994-a100-2cbfaee5b853 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:38.633298 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1381/2769] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:39:38.906979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a22a07-24bb-47e1-ae79-49eb614ed8fc None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:38.908469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-92a22a07-24bb-47e1-ae79-49eb614ed8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:38.922469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a22a07-24bb-47e1-ae79-49eb614ed8fc None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:38.927899 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:38.928561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:38.928836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:38.929109 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] PATH_INFO: `/v3/projects/d45e5a66699e4a8fbe4322b09360f11c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:38.930406 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d45e5a66699e4a8fbe4322b09360f11c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:38.932279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:38.979062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd45e5a66699e4a8fbe4322b09360f11c', 'request_id': 'req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:38.985058 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd45e5a66699e4a8fbe4322b09360f11c', 'request_id': 'req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:38.991745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd45e5a66699e4a8fbe4322b09360f11c', 'request_id': 'req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:39.006517 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] The token cache is being invalidate because project d45e5a66699e4a8fbe4322b09360f11c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:39:39.006824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:39.007467 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:39:39.009070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1389/2770] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:39:38 2022] DELETE /identity/v3/projects/d45e5a66699e4a8fbe4322b09360f11c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:39:45.629534 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c720745-ad7e-425e-a8c4-1ad59667e72a None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:45.630100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c720745-ad7e-425e-a8c4-1ad59667e72a None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:45.630100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c720745-ad7e-425e-a8c4-1ad59667e72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:45.650882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-0c720745-ad7e-425e-a8c4-1ad59667e72a None None] MFA Rules not processed for user `5ae302ea931a42b5938884e79c2ae381`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:39:45.652083 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0c720745-ad7e-425e-a8c4-1ad59667e72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:45.710168 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1382/2771] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:39:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:39:49.003479 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=66,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:54.156357 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:54.157880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:54.219085 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab9bab-0613-4cbf-a86f-0d9fef309b3c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:54.223340 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd87e6-fdbc-41ed-bb3e-8aea15f6430c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:54.223945 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd87e6-fdbc-41ed-bb3e-8aea15f6430c service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:54.224201 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd87e6-fdbc-41ed-bb3e-8aea15f6430c service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:54.224492 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd87e6-fdbc-41ed-bb3e-8aea15f6430c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:54.226353 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a0bd87e6-fdbc-41ed-bb3e-8aea15f6430c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:54.298514 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd87e6-fdbc-41ed-bb3e-8aea15f6430c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:54.300006 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd87e6-fdbc-41ed-bb3e-8aea15f6430c service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:54.310677 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1390/2772] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:39:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:39:55.702994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=29 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:39:58.398125 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c720745-ad7e-425e-a8c4-1ad59667e72a None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:39:58.399671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0c720745-ad7e-425e-a8c4-1ad59667e72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:58.495680 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c720745-ad7e-425e-a8c4-1ad59667e72a None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:39:58.501228 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ac7dc-21ed-429c-a832-b7107c11ba75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784b6147de91435292e4cf3940bcad9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:39:58.501919 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ac7dc-21ed-429c-a832-b7107c11ba75 service placement] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:39:58.502262 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ac7dc-21ed-429c-a832-b7107c11ba75 service placement] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:39:58.502627 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ac7dc-21ed-429c-a832-b7107c11ba75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:39:58.505009 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-fb8ac7dc-21ed-429c-a832-b7107c11ba75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:39:58.518392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ac7dc-21ed-429c-a832-b7107c11ba75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:39:58.520193 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ac7dc-21ed-429c-a832-b7107c11ba75 service placement] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:39:58.535053 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1383/2773] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:39:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:40:04.305475 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=50 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:08.529684 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=41 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:11.911267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd87e6-fdbc-41ed-bb3e-8aea15f6430c service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:11.912808 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a0bd87e6-fdbc-41ed-bb3e-8aea15f6430c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:11.924156 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd87e6-fdbc-41ed-bb3e-8aea15f6430c service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:11.928721 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef9b2f-65ff-4b6d-99ad-c8c940a814b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:11.929319 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef9b2f-65ff-4b6d-99ad-c8c940a814b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:11.929620 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef9b2f-65ff-4b6d-99ad-c8c940a814b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:11.929891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef9b2f-65ff-4b6d-99ad-c8c940a814b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:11.931573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-49ef9b2f-65ff-4b6d-99ad-c8c940a814b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:11.996029 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef9b2f-65ff-4b6d-99ad-c8c940a814b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:11.997726 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef9b2f-65ff-4b6d-99ad-c8c940a814b5 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:12.007858 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1391/2774] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:40:15.173577 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ac7dc-21ed-429c-a832-b7107c11ba75 service placement] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:15.187465 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ac7dc-21ed-429c-a832-b7107c11ba75 service placement] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:15.193122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893355f7-0935-43d7-ba4d-c7f60de87f4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784b6147de91435292e4cf3940bcad9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:15.194043 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893355f7-0935-43d7-ba4d-c7f60de87f4c service placement] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:15.194318 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893355f7-0935-43d7-ba4d-c7f60de87f4c service placement] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:15.194609 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893355f7-0935-43d7-ba4d-c7f60de87f4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:15.196356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-893355f7-0935-43d7-ba4d-c7f60de87f4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:15.210220 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893355f7-0935-43d7-ba4d-c7f60de87f4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:15.212952 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893355f7-0935-43d7-ba4d-c7f60de87f4c service placement] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:15.229894 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1384/2775] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:40:15.265563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef9b2f-65ff-4b6d-99ad-c8c940a814b5 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:15.268154 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-49ef9b2f-65ff-4b6d-99ad-c8c940a814b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:15.283286 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef9b2f-65ff-4b6d-99ad-c8c940a814b5 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:15.288971 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d49174-9284-49dc-b235-8869339c601a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:15.289592 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d49174-9284-49dc-b235-8869339c601a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:15.290173 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d49174-9284-49dc-b235-8869339c601a admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:15.290594 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d49174-9284-49dc-b235-8869339c601a admin admin] PATH_INFO: `/v3/users/f8bd900e231242c3ae67432d26d089da` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:15.291719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d49174-9284-49dc-b235-8869339c601a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8bd900e231242c3ae67432d26d089da)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:15.293553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d49174-9284-49dc-b235-8869339c601a admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:15.349080 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-40d49174-9284-49dc-b235-8869339c601a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8bd900e231242c3ae67432d26d089da'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:15.349970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-40d49174-9284-49dc-b235-8869339c601a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8bd900e231242c3ae67432d26d089da'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:15.350744 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-40d49174-9284-49dc-b235-8869339c601a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8bd900e231242c3ae67432d26d089da'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:15.357835 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-40d49174-9284-49dc-b235-8869339c601a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8bd900e231242c3ae67432d26d089da'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:15.370995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-40d49174-9284-49dc-b235-8869339c601a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8bd900e231242c3ae67432d26d089da'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:15.386802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1392/2776] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:40:15 2022] DELETE /identity/v3/users/f8bd900e231242c3ae67432d26d089da => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:15.572211 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893355f7-0935-43d7-ba4d-c7f60de87f4c service placement] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:15.601498 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893355f7-0935-43d7-ba4d-c7f60de87f4c service placement] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:15.664614 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:15.665290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:15.665572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:15.665840 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc admin admin] PATH_INFO: `/v3/projects/3fca9208950f4d1a92bd52b4e5f5b2a1` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:15.667008 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fca9208950f4d1a92bd52b4e5f5b2a1)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:15.668927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:15.717978 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fca9208950f4d1a92bd52b4e5f5b2a1', 'request_id': 'req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:15.724130 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fca9208950f4d1a92bd52b4e5f5b2a1', 'request_id': 'req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:15.731474 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fca9208950f4d1a92bd52b4e5f5b2a1', 'request_id': 'req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:15.741642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc admin admin] The token cache is being invalidate because project 3fca9208950f4d1a92bd52b4e5f5b2a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:40:15.741922 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:15.742555 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ff7f0b20-0f1d-40da-a8c6-f7d03ed449fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:15.744074 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1385/2777] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:40:15 2022] DELETE /identity/v3/projects/3fca9208950f4d1a92bd52b4e5f5b2a1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:15.749002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb3015-c34f-48ba-b205-c31b0d9d2a85 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:15.749283 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb3015-c34f-48ba-b205-c31b0d9d2a85 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:15.749549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb3015-c34f-48ba-b205-c31b0d9d2a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:15.789868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-9bbb3015-c34f-48ba-b205-c31b0d9d2a85 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:15.791142 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-9bbb3015-c34f-48ba-b205-c31b0d9d2a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:15.842232 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:15.864306 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1393/2778] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:15.872273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc5871-80b6-4084-a69d-beb94ab893ba None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:15.872552 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc5871-80b6-4084-a69d-beb94ab893ba None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:15.873014 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc5871-80b6-4084-a69d-beb94ab893ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:15.895557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-69bc5871-80b6-4084-a69d-beb94ab893ba None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:15.896990 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-69bc5871-80b6-4084-a69d-beb94ab893ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:15.908889 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1386/2779] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:40:15.910245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb3015-c34f-48ba-b205-c31b0d9d2a85 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:15.921322 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb3015-c34f-48ba-b205-c31b0d9d2a85 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:15.926130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc381a78-88d0-4b5d-bfb7-71abf39451b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:15.926775 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc381a78-88d0-4b5d-bfb7-71abf39451b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:15.927055 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc381a78-88d0-4b5d-bfb7-71abf39451b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:15.927322 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc381a78-88d0-4b5d-bfb7-71abf39451b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:15.927973 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc381a78-88d0-4b5d-bfb7-71abf39451b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:40:15.928365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc381a78-88d0-4b5d-bfb7-71abf39451b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:15.929981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc381a78-88d0-4b5d-bfb7-71abf39451b1 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:15.948871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1394/2780] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:40:15.949539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc5871-80b6-4084-a69d-beb94ab893ba None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:15.959052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc5871-80b6-4084-a69d-beb94ab893ba None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:15.964822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7cf53-4be3-4683-9e6d-afdc0f0ab2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:15.965443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7cf53-4be3-4683-9e6d-afdc0f0ab2fe admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:15.965787 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7cf53-4be3-4683-9e6d-afdc0f0ab2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:15.966055 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7cf53-4be3-4683-9e6d-afdc0f0ab2fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:15.966941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7cf53-4be3-4683-9e6d-afdc0f0ab2fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:15.968933 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7cf53-4be3-4683-9e6d-afdc0f0ab2fe admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:15.997053 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1387/2781] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:40:15 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:40:16.001298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc381a78-88d0-4b5d-bfb7-71abf39451b1 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:16.011668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc381a78-88d0-4b5d-bfb7-71abf39451b1 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:16.071377 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c547b-e6b9-4c39-a83b-432434d55d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:16.071983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c547b-e6b9-4c39-a83b-432434d55d83 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:16.072264 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c547b-e6b9-4c39-a83b-432434d55d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:16.072536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c547b-e6b9-4c39-a83b-432434d55d83 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:16.073155 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c547b-e6b9-4c39-a83b-432434d55d83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:16.075138 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c547b-e6b9-4c39-a83b-432434d55d83 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:16.099839 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1395/2782] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:40:16 2022] POST /identity/v3/users => generated 302 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:40:16.104227 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7cf53-4be3-4683-9e6d-afdc0f0ab2fe admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:16.115105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7cf53-4be3-4683-9e6d-afdc0f0ab2fe admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:16.120868 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3c9ba-4f87-4597-9d70-4ed1248a6799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:16.121508 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3c9ba-4f87-4597-9d70-4ed1248a6799 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:16.121790 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3c9ba-4f87-4597-9d70-4ed1248a6799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:16.122063 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3c9ba-4f87-4597-9d70-4ed1248a6799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:16.122754 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d3c9ba-4f87-4597-9d70-4ed1248a6799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:16.124390 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d3c9ba-4f87-4597-9d70-4ed1248a6799 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:16.134384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1388/2783] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:16.138591 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c547b-e6b9-4c39-a83b-432434d55d83 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:16.148876 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c547b-e6b9-4c39-a83b-432434d55d83 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:16.153816 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b656a1-69e2-4d99-80f9-6aa92688c859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:16.154441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b656a1-69e2-4d99-80f9-6aa92688c859 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:16.154755 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b656a1-69e2-4d99-80f9-6aa92688c859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:16.155027 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b656a1-69e2-4d99-80f9-6aa92688c859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:16.155666 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b656a1-69e2-4d99-80f9-6aa92688c859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:16.157299 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b656a1-69e2-4d99-80f9-6aa92688c859 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:16.168546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1396/2784] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:16.172596 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3c9ba-4f87-4597-9d70-4ed1248a6799 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:16.182530 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3c9ba-4f87-4597-9d70-4ed1248a6799 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:16.188077 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d996293-7162-41f1-93b6-895a73572d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:16.188757 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d996293-7162-41f1-93b6-895a73572d9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:16.189035 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d996293-7162-41f1-93b6-895a73572d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:16.189311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d996293-7162-41f1-93b6-895a73572d9f admin admin] PATH_INFO: `/v3/projects/a1e476d4ddb04c46b25ddced1c4e2901/users/bf6249c991354ed8976103a1d80d062d/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:16.204504 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d996293-7162-41f1-93b6-895a73572d9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1e476d4ddb04c46b25ddced1c4e2901, user_id=bf6249c991354ed8976103a1d80d062d, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:16.207243 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d996293-7162-41f1-93b6-895a73572d9f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:16.230882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1389/2785] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:40:16 2022] PUT /identity/v3/projects/a1e476d4ddb04c46b25ddced1c4e2901/users/bf6249c991354ed8976103a1d80d062d/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:40:16.237120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec38bb-42e5-41b6-84f1-9fc88d20f46b None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:16.237402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec38bb-42e5-41b6-84f1-9fc88d20f46b None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:16.237665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec38bb-42e5-41b6-84f1-9fc88d20f46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:16.266485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-07ec38bb-42e5-41b6-84f1-9fc88d20f46b None None] MFA Rules not processed for user `bf6249c991354ed8976103a1d80d062d`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:16.267756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-07ec38bb-42e5-41b6-84f1-9fc88d20f46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:16.335415 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1397/2786] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:16 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:40:16.339938 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d996293-7162-41f1-93b6-895a73572d9f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:16.355375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:16.408666 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d996293-7162-41f1-93b6-895a73572d9f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:16.413906 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5d69f-7799-4364-bd1a-b5ce650de851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:16.414545 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5d69f-7799-4364-bd1a-b5ce650de851 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:16.414829 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5d69f-7799-4364-bd1a-b5ce650de851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:16.415084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5d69f-7799-4364-bd1a-b5ce650de851 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:16.415717 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5d69f-7799-4364-bd1a-b5ce650de851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:16.417603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5d69f-7799-4364-bd1a-b5ce650de851 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:16.444918 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1390/2787] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:40:16 2022] POST /identity/v3/projects => generated 403 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:40:16.449046 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec38bb-42e5-41b6-84f1-9fc88d20f46b None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:16.460377 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec38bb-42e5-41b6-84f1-9fc88d20f46b None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:16.505247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:16.519397 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadf07b-47ab-405e-bfd0-8505421ac298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:16.520106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadf07b-47ab-405e-bfd0-8505421ac298 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:16.520441 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadf07b-47ab-405e-bfd0-8505421ac298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:16.520770 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadf07b-47ab-405e-bfd0-8505421ac298 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:16.521455 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadf07b-47ab-405e-bfd0-8505421ac298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:16.523556 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadf07b-47ab-405e-bfd0-8505421ac298 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:16.549744 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1398/2788] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:40:16 2022] POST /identity/v3/users => generated 308 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:40:16.551017 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5d69f-7799-4364-bd1a-b5ce650de851 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:16.561917 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5d69f-7799-4364-bd1a-b5ce650de851 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:16.568072 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cda71-5bd8-492d-bed1-048afe85ee77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:16.568834 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cda71-5bd8-492d-bed1-048afe85ee77 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:16.569184 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cda71-5bd8-492d-bed1-048afe85ee77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:16.569459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cda71-5bd8-492d-bed1-048afe85ee77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:16.570107 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958cda71-5bd8-492d-bed1-048afe85ee77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:16.571759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958cda71-5bd8-492d-bed1-048afe85ee77 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:16.583435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1391/2789] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:16.588126 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadf07b-47ab-405e-bfd0-8505421ac298 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:16.598852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadf07b-47ab-405e-bfd0-8505421ac298 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:16.604075 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce5780-2d0e-47c6-9e26-0a9bb35ba05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:16.604731 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce5780-2d0e-47c6-9e26-0a9bb35ba05c admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:16.605002 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce5780-2d0e-47c6-9e26-0a9bb35ba05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:16.605271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce5780-2d0e-47c6-9e26-0a9bb35ba05c admin admin] PATH_INFO: `/v3/projects/f224241bb94b46b2866408b45f1f1e43/users/4439b655d04b47f88f55b6d6b8892f90/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:16.620891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce5780-2d0e-47c6-9e26-0a9bb35ba05c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f224241bb94b46b2866408b45f1f1e43, user_id=4439b655d04b47f88f55b6d6b8892f90, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:16.624617 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce5780-2d0e-47c6-9e26-0a9bb35ba05c admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:16.644487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1399/2790] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:40:16 2022] PUT /identity/v3/projects/f224241bb94b46b2866408b45f1f1e43/users/4439b655d04b47f88f55b6d6b8892f90/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:16.649473 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cda71-5bd8-492d-bed1-048afe85ee77 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:16.659946 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cda71-5bd8-492d-bed1-048afe85ee77 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:16.721759 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ccff1-1950-4db7-84fc-200888dea04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:16.722551 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ccff1-1950-4db7-84fc-200888dea04b admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:16.722909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ccff1-1950-4db7-84fc-200888dea04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:16.723273 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ccff1-1950-4db7-84fc-200888dea04b admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:16.724236 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ccff1-1950-4db7-84fc-200888dea04b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:16.726803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ccff1-1950-4db7-84fc-200888dea04b admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:16.737065 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1392/2791] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:16.741709 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce5780-2d0e-47c6-9e26-0a9bb35ba05c admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:16.752533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce5780-2d0e-47c6-9e26-0a9bb35ba05c admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:16.757820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac133e83-b088-404d-9324-d97896df1a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:16.758472 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac133e83-b088-404d-9324-d97896df1a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:16.758785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac133e83-b088-404d-9324-d97896df1a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:16.759074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac133e83-b088-404d-9324-d97896df1a27 admin admin] PATH_INFO: `/v3/projects/f224241bb94b46b2866408b45f1f1e43/users/4439b655d04b47f88f55b6d6b8892f90/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:16.761012 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac133e83-b088-404d-9324-d97896df1a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f224241bb94b46b2866408b45f1f1e43, user_id=4439b655d04b47f88f55b6d6b8892f90, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:16.763871 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac133e83-b088-404d-9324-d97896df1a27 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:16.774543 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1400/2792] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:40:16 2022] PUT /identity/v3/projects/f224241bb94b46b2866408b45f1f1e43/users/4439b655d04b47f88f55b6d6b8892f90/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:16.784740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e92b2-361f-4ef6-b45c-2ea93a8851f5 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:16.785052 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e92b2-361f-4ef6-b45c-2ea93a8851f5 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:16.785690 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e92b2-361f-4ef6-b45c-2ea93a8851f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:16.815029 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-bf3e92b2-361f-4ef6-b45c-2ea93a8851f5 None None] MFA Rules not processed for user `4439b655d04b47f88f55b6d6b8892f90`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:16.816272 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-bf3e92b2-361f-4ef6-b45c-2ea93a8851f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:16.886431 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1393/2793] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:16 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:16.897511 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac133e83-b088-404d-9324-d97896df1a27 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:16.899548 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-ac133e83-b088-404d-9324-d97896df1a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:16.961600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac133e83-b088-404d-9324-d97896df1a27 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:16.965788 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186574bd-46a9-403d-800a-c3bf9d0ef237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:16.966390 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186574bd-46a9-403d-800a-c3bf9d0ef237 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:16.966696 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186574bd-46a9-403d-800a-c3bf9d0ef237 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:16.966964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186574bd-46a9-403d-800a-c3bf9d0ef237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:16.976410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186574bd-46a9-403d-800a-c3bf9d0ef237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:16.977819 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186574bd-46a9-403d-800a-c3bf9d0ef237 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:16.987365 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1401/2794] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:16 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:40:17.127105 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e92b2-361f-4ef6-b45c-2ea93a8851f5 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:17.139267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:17.193741 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e92b2-361f-4ef6-b45c-2ea93a8851f5 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:17.199490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f626f36-da15-4c58-a65a-118e83e76f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:17.200123 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f626f36-da15-4c58-a65a-118e83e76f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:17.200398 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f626f36-da15-4c58-a65a-118e83e76f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:17.200671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f626f36-da15-4c58-a65a-118e83e76f46 admin admin] PATH_INFO: `/v3/users/bf6249c991354ed8976103a1d80d062d` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:17.201808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f626f36-da15-4c58-a65a-118e83e76f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf6249c991354ed8976103a1d80d062d)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:17.203821 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f626f36-da15-4c58-a65a-118e83e76f46 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:17.254282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2f626f36-da15-4c58-a65a-118e83e76f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf6249c991354ed8976103a1d80d062d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:17.254890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2f626f36-da15-4c58-a65a-118e83e76f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf6249c991354ed8976103a1d80d062d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:17.255505 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2f626f36-da15-4c58-a65a-118e83e76f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf6249c991354ed8976103a1d80d062d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:17.262360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2f626f36-da15-4c58-a65a-118e83e76f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf6249c991354ed8976103a1d80d062d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:17.274434 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-2f626f36-da15-4c58-a65a-118e83e76f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf6249c991354ed8976103a1d80d062d'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:17.288737 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1394/2795] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:40:17 2022] DELETE /identity/v3/users/bf6249c991354ed8976103a1d80d062d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:17.293470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5df3c-06d7-4c48-9ae3-ef3834605c00 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:17.293956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5df3c-06d7-4c48-9ae3-ef3834605c00 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:17.294074 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5df3c-06d7-4c48-9ae3-ef3834605c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:17.313572 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-fdf5df3c-06d7-4c48-9ae3-ef3834605c00 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:17.314799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-fdf5df3c-06d7-4c48-9ae3-ef3834605c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:17.377120 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1402/2796] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:17.388736 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f626f36-da15-4c58-a65a-118e83e76f46 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:17.390264 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2f626f36-da15-4c58-a65a-118e83e76f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:17.456095 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f626f36-da15-4c58-a65a-118e83e76f46 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:17.460437 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aadf80-b249-42ab-929d-59cb10de013a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:17.461044 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aadf80-b249-42ab-929d-59cb10de013a service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:17.461317 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aadf80-b249-42ab-929d-59cb10de013a service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:17.461580 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aadf80-b249-42ab-929d-59cb10de013a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:17.471619 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aadf80-b249-42ab-929d-59cb10de013a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:17.473042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aadf80-b249-42ab-929d-59cb10de013a service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:17.482836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1395/2797] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:40:17.799098 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5df3c-06d7-4c48-9ae3-ef3834605c00 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:17.800704 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-fdf5df3c-06d7-4c48-9ae3-ef3834605c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:17.812852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5df3c-06d7-4c48-9ae3-ef3834605c00 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:17.817528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:17.818161 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:17.818435 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:17.818799 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] PATH_INFO: `/v3/projects/a1e476d4ddb04c46b25ddced1c4e2901` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:17.819988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1e476d4ddb04c46b25ddced1c4e2901)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:17.821902 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:17.863022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:17.872532 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1e476d4ddb04c46b25ddced1c4e2901', 'request_id': 'req-8eab3395-ece6-42c7-977a-f53f222f24a5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:17.878890 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1e476d4ddb04c46b25ddced1c4e2901', 'request_id': 'req-8eab3395-ece6-42c7-977a-f53f222f24a5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:17.884312 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1e476d4ddb04c46b25ddced1c4e2901', 'request_id': 'req-8eab3395-ece6-42c7-977a-f53f222f24a5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:17.898194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] The token cache is being invalidate because project a1e476d4ddb04c46b25ddced1c4e2901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:40:17.898467 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:17.899182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:17.900665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1403/2798] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:40:17 2022] DELETE /identity/v3/projects/a1e476d4ddb04c46b25ddced1c4e2901 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:17.904670 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aadf80-b249-42ab-929d-59cb10de013a service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:17.915467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aadf80-b249-42ab-929d-59cb10de013a service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:17.944824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:17.975005 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868e795-5cb4-4f3a-97c3-2226de0efaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:17.975662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868e795-5cb4-4f3a-97c3-2226de0efaa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:17.975932 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868e795-5cb4-4f3a-97c3-2226de0efaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:17.976196 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868e795-5cb4-4f3a-97c3-2226de0efaa9 admin admin] PATH_INFO: `/v3/users/4439b655d04b47f88f55b6d6b8892f90` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:17.977360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868e795-5cb4-4f3a-97c3-2226de0efaa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4439b655d04b47f88f55b6d6b8892f90)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:17.979308 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868e795-5cb4-4f3a-97c3-2226de0efaa9 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:18.026929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1868e795-5cb4-4f3a-97c3-2226de0efaa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4439b655d04b47f88f55b6d6b8892f90'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:18.027582 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1868e795-5cb4-4f3a-97c3-2226de0efaa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4439b655d04b47f88f55b6d6b8892f90'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:18.028152 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1868e795-5cb4-4f3a-97c3-2226de0efaa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4439b655d04b47f88f55b6d6b8892f90'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:18.034768 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1868e795-5cb4-4f3a-97c3-2226de0efaa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4439b655d04b47f88f55b6d6b8892f90'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:18.046656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1868e795-5cb4-4f3a-97c3-2226de0efaa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4439b655d04b47f88f55b6d6b8892f90'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:18.060419 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1396/2799] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:40:17 2022] DELETE /identity/v3/users/4439b655d04b47f88f55b6d6b8892f90 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:18.474111 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:18.474924 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:18.496256 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab3395-ece6-42c7-977a-f53f222f24a5 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:18.572144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ffd1f-4659-4e19-8d31-6065cd84add4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:18.572888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ffd1f-4659-4e19-8d31-6065cd84add4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:18.573198 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ffd1f-4659-4e19-8d31-6065cd84add4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:18.573568 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ffd1f-4659-4e19-8d31-6065cd84add4 admin admin] PATH_INFO: `/v3/projects/f224241bb94b46b2866408b45f1f1e43` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:18.574888 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ffd1f-4659-4e19-8d31-6065cd84add4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f224241bb94b46b2866408b45f1f1e43)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:18.577666 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ffd1f-4659-4e19-8d31-6065cd84add4 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:18.629901 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f8ffd1f-4659-4e19-8d31-6065cd84add4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f224241bb94b46b2866408b45f1f1e43', 'request_id': 'req-4f8ffd1f-4659-4e19-8d31-6065cd84add4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:18.636660 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f8ffd1f-4659-4e19-8d31-6065cd84add4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f224241bb94b46b2866408b45f1f1e43', 'request_id': 'req-4f8ffd1f-4659-4e19-8d31-6065cd84add4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:18.642402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f8ffd1f-4659-4e19-8d31-6065cd84add4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f224241bb94b46b2866408b45f1f1e43', 'request_id': 'req-4f8ffd1f-4659-4e19-8d31-6065cd84add4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:18.656380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f8ffd1f-4659-4e19-8d31-6065cd84add4 admin admin] The token cache is being invalidate because project f224241bb94b46b2866408b45f1f1e43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:40:18.656752 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f8ffd1f-4659-4e19-8d31-6065cd84add4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:18.657422 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-4f8ffd1f-4659-4e19-8d31-6065cd84add4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:18.659425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1404/2800] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:40:18 2022] DELETE /identity/v3/projects/f224241bb94b46b2866408b45f1f1e43 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:18.664562 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad9902-aed6-4415-b18c-8d57574df27f None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:18.665057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad9902-aed6-4415-b18c-8d57574df27f None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:18.665180 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad9902-aed6-4415-b18c-8d57574df27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:18.687907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-48ad9902-aed6-4415-b18c-8d57574df27f None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:18.689190 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-48ad9902-aed6-4415-b18c-8d57574df27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:18.767213 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1397/2801] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:18.776719 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f9b0a-acec-460d-b51b-6723283f15de None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:18.777250 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f9b0a-acec-460d-b51b-6723283f15de None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:18.777690 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f9b0a-acec-460d-b51b-6723283f15de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:18.799328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5b8f9b0a-acec-460d-b51b-6723283f15de None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:18.800604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5b8f9b0a-acec-460d-b51b-6723283f15de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:18.808918 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1405/2802] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:18.813218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad9902-aed6-4415-b18c-8d57574df27f None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:18.823290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad9902-aed6-4415-b18c-8d57574df27f None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:18.830028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644054d-256f-4726-be86-d8ca7f794aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:18.830691 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644054d-256f-4726-be86-d8ca7f794aab admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:18.830953 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644054d-256f-4726-be86-d8ca7f794aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:18.831221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644054d-256f-4726-be86-d8ca7f794aab admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:18.831868 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644054d-256f-4726-be86-d8ca7f794aab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:40:18.832441 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644054d-256f-4726-be86-d8ca7f794aab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:18.834221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644054d-256f-4726-be86-d8ca7f794aab admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:18.846796 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1398/2803] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:40:18.851578 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f9b0a-acec-460d-b51b-6723283f15de None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:18.861671 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f9b0a-acec-460d-b51b-6723283f15de None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:18.866598 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964567b-dae3-4c24-9311-0e6b0b4307aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:18.867213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964567b-dae3-4c24-9311-0e6b0b4307aa admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:18.867495 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964567b-dae3-4c24-9311-0e6b0b4307aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:18.867767 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964567b-dae3-4c24-9311-0e6b0b4307aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:18.868405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964567b-dae3-4c24-9311-0e6b0b4307aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:18.870344 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964567b-dae3-4c24-9311-0e6b0b4307aa admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:18.898758 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1406/2804] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:40:18 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:40:18.903031 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644054d-256f-4726-be86-d8ca7f794aab admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:18.913836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644054d-256f-4726-be86-d8ca7f794aab admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:18.948976 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:18.972624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b39702-4336-41cc-b91f-aa8ec04a4b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:18.973326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b39702-4336-41cc-b91f-aa8ec04a4b1d admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:18.973501 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b39702-4336-41cc-b91f-aa8ec04a4b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:18.973771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b39702-4336-41cc-b91f-aa8ec04a4b1d admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:18.974403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b39702-4336-41cc-b91f-aa8ec04a4b1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:18.976375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b39702-4336-41cc-b91f-aa8ec04a4b1d admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:19.001387 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1399/2805] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:40:18 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:40:19.005734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964567b-dae3-4c24-9311-0e6b0b4307aa admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:19.020409 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964567b-dae3-4c24-9311-0e6b0b4307aa admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:19.022589 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1477ea7-2a2c-447a-97d2-9878feb627cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:19.023217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1477ea7-2a2c-447a-97d2-9878feb627cc admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:19.023487 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1477ea7-2a2c-447a-97d2-9878feb627cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:19.023748 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1477ea7-2a2c-447a-97d2-9878feb627cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:19.024383 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1477ea7-2a2c-447a-97d2-9878feb627cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:19.030754 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1477ea7-2a2c-447a-97d2-9878feb627cc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:19.037099 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1407/2806] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:19.041481 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b39702-4336-41cc-b91f-aa8ec04a4b1d admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:19.054797 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b39702-4336-41cc-b91f-aa8ec04a4b1d admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:19.060623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d691f8-3b90-46ce-8151-ec22e4d1b8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:19.061247 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d691f8-3b90-46ce-8151-ec22e4d1b8da admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:19.061519 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d691f8-3b90-46ce-8151-ec22e4d1b8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:19.061781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d691f8-3b90-46ce-8151-ec22e4d1b8da admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:19.062421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d691f8-3b90-46ce-8151-ec22e4d1b8da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:19.064057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d691f8-3b90-46ce-8151-ec22e4d1b8da admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:19.076025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1400/2807] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:19.084639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1477ea7-2a2c-447a-97d2-9878feb627cc admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:19.095575 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1477ea7-2a2c-447a-97d2-9878feb627cc admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:19.101089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f369e74-b593-4f76-b6f5-a5ac3ece0965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:19.102026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f369e74-b593-4f76-b6f5-a5ac3ece0965 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:19.102352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f369e74-b593-4f76-b6f5-a5ac3ece0965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:19.102732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f369e74-b593-4f76-b6f5-a5ac3ece0965 admin admin] PATH_INFO: `/v3/projects/987ae45b30094b119441849ca3df098b/users/0d66a5d034f04f928c348267d35d0cdf/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:19.117056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:19.120399 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f369e74-b593-4f76-b6f5-a5ac3ece0965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=987ae45b30094b119441849ca3df098b, user_id=0d66a5d034f04f928c348267d35d0cdf, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:19.123346 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f369e74-b593-4f76-b6f5-a5ac3ece0965 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:19.139195 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1408/2808] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:40:19 2022] PUT /identity/v3/projects/987ae45b30094b119441849ca3df098b/users/0d66a5d034f04f928c348267d35d0cdf/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:19.151042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4162c3-14db-4af5-b63a-70fdd8e0b53d None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:19.151736 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4162c3-14db-4af5-b63a-70fdd8e0b53d None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:19.152078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4162c3-14db-4af5-b63a-70fdd8e0b53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:19.201270 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-0c4162c3-14db-4af5-b63a-70fdd8e0b53d None None] MFA Rules not processed for user `0d66a5d034f04f928c348267d35d0cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:19.202485 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0c4162c3-14db-4af5-b63a-70fdd8e0b53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:19.269103 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1401/2809] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:19.273934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f369e74-b593-4f76-b6f5-a5ac3ece0965 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:19.339756 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f369e74-b593-4f76-b6f5-a5ac3ece0965 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:19.345684 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a525136-056e-46bd-bc94-73623f0e8b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:19.346290 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525136-056e-46bd-bc94-73623f0e8b9d admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:19.346560 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525136-056e-46bd-bc94-73623f0e8b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:19.346867 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525136-056e-46bd-bc94-73623f0e8b9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:19.347490 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a525136-056e-46bd-bc94-73623f0e8b9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:19.349394 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a525136-056e-46bd-bc94-73623f0e8b9d admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:19.379725 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1409/2810] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:40:19 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:40:19.383886 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4162c3-14db-4af5-b63a-70fdd8e0b53d None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:19.394910 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4162c3-14db-4af5-b63a-70fdd8e0b53d None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:19.452449 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ce00f-2812-4c2f-97b7-10f574381667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:19.453337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ce00f-2812-4c2f-97b7-10f574381667 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:19.453819 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ce00f-2812-4c2f-97b7-10f574381667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:19.454240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ce00f-2812-4c2f-97b7-10f574381667 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:19.455073 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ce00f-2812-4c2f-97b7-10f574381667 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:19.457244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ce00f-2812-4c2f-97b7-10f574381667 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:19.479781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1402/2811] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:40:19 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:40:19.484048 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a525136-056e-46bd-bc94-73623f0e8b9d admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:19.495274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a525136-056e-46bd-bc94-73623f0e8b9d admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:19.500303 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b11318-eed5-47b3-813d-5708134f4a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:19.500901 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b11318-eed5-47b3-813d-5708134f4a42 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:19.501174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b11318-eed5-47b3-813d-5708134f4a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:19.501439 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b11318-eed5-47b3-813d-5708134f4a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:19.502076 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b11318-eed5-47b3-813d-5708134f4a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:19.503908 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b11318-eed5-47b3-813d-5708134f4a42 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:19.516082 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1410/2812] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:19.521106 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ce00f-2812-4c2f-97b7-10f574381667 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:19.531476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ce00f-2812-4c2f-97b7-10f574381667 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:19.536607 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8358b4-5adb-4f79-9b93-113763a3aa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:19.537259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8358b4-5adb-4f79-9b93-113763a3aa56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:19.537535 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8358b4-5adb-4f79-9b93-113763a3aa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:19.537802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8358b4-5adb-4f79-9b93-113763a3aa56 admin admin] PATH_INFO: `/v3/projects/738fd57500434dfaa46c095427fb71d8/users/9774137b6b254da1bbeeddad29a26d84/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:19.552388 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8358b4-5adb-4f79-9b93-113763a3aa56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=738fd57500434dfaa46c095427fb71d8, user_id=9774137b6b254da1bbeeddad29a26d84, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:19.555147 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8358b4-5adb-4f79-9b93-113763a3aa56 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:19.570091 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1403/2813] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:40:19 2022] PUT /identity/v3/projects/738fd57500434dfaa46c095427fb71d8/users/9774137b6b254da1bbeeddad29a26d84/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:40:19.574118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b11318-eed5-47b3-813d-5708134f4a42 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:19.584380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b11318-eed5-47b3-813d-5708134f4a42 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:19.642570 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18136d-04ac-4f5d-8cce-17bf3a3b5a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:19.643102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18136d-04ac-4f5d-8cce-17bf3a3b5a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:19.643376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18136d-04ac-4f5d-8cce-17bf3a3b5a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:19.643642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18136d-04ac-4f5d-8cce-17bf3a3b5a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:19.644274 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18136d-04ac-4f5d-8cce-17bf3a3b5a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:19.645868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18136d-04ac-4f5d-8cce-17bf3a3b5a13 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:19.655947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1411/2814] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:19.660693 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8358b4-5adb-4f79-9b93-113763a3aa56 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:19.671260 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8358b4-5adb-4f79-9b93-113763a3aa56 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:19.676199 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f653837-736d-49e7-8d1c-ca28e285af80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:19.676845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f653837-736d-49e7-8d1c-ca28e285af80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:19.677120 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f653837-736d-49e7-8d1c-ca28e285af80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:19.677389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f653837-736d-49e7-8d1c-ca28e285af80 admin admin] PATH_INFO: `/v3/projects/738fd57500434dfaa46c095427fb71d8/users/9774137b6b254da1bbeeddad29a26d84/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:19.679435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f653837-736d-49e7-8d1c-ca28e285af80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=738fd57500434dfaa46c095427fb71d8, user_id=9774137b6b254da1bbeeddad29a26d84, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:19.682067 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f653837-736d-49e7-8d1c-ca28e285af80 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:19.687586 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:19.693993 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1404/2815] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:40:19 2022] PUT /identity/v3/projects/738fd57500434dfaa46c095427fb71d8/users/9774137b6b254da1bbeeddad29a26d84/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:19.704348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064a2f4-e0be-478c-a83d-d948eb984b50 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:19.704633 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064a2f4-e0be-478c-a83d-d948eb984b50 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:19.704908 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064a2f4-e0be-478c-a83d-d948eb984b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:19.734187 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3064a2f4-e0be-478c-a83d-d948eb984b50 None None] MFA Rules not processed for user `9774137b6b254da1bbeeddad29a26d84`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:19.735448 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3064a2f4-e0be-478c-a83d-d948eb984b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:19.787278 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:19.809314 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1412/2816] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:19.821204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f653837-736d-49e7-8d1c-ca28e285af80 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:19.823124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-6f653837-736d-49e7-8d1c-ca28e285af80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:19.882415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f653837-736d-49e7-8d1c-ca28e285af80 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:19.886936 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638c49a-719b-49a2-890a-b1605ef9c6d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:19.887775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638c49a-719b-49a2-890a-b1605ef9c6d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:19.888038 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638c49a-719b-49a2-890a-b1605ef9c6d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:19.888298 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638c49a-719b-49a2-890a-b1605ef9c6d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:19.897931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638c49a-719b-49a2-890a-b1605ef9c6d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:19.899335 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638c49a-719b-49a2-890a-b1605ef9c6d9 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:19.912383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1405/2817] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:40:19.986933 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064a2f4-e0be-478c-a83d-d948eb984b50 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:19.996586 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064a2f4-e0be-478c-a83d-d948eb984b50 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:20.000205 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87aeab2-0dde-4491-9e13-a0490afc4675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:20.000814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87aeab2-0dde-4491-9e13-a0490afc4675 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:20.001086 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87aeab2-0dde-4491-9e13-a0490afc4675 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:20.001352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87aeab2-0dde-4491-9e13-a0490afc4675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:20.058228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87aeab2-0dde-4491-9e13-a0490afc4675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:20.059752 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87aeab2-0dde-4491-9e13-a0490afc4675 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:20.069318 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1413/2818] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:40:29.904539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=27 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:30.064638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=34 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:31.246832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638c49a-719b-49a2-890a-b1605ef9c6d9 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:31.311362 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638c49a-719b-49a2-890a-b1605ef9c6d9 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:31.316434 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a3ae2-41a7-4e1b-86bc-22b987257ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:31.317078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a3ae2-41a7-4e1b-86bc-22b987257ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:31.317358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a3ae2-41a7-4e1b-86bc-22b987257ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:31.317623 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a3ae2-41a7-4e1b-86bc-22b987257ff8 admin admin] PATH_INFO: `/v3/users/0d66a5d034f04f928c348267d35d0cdf` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:31.318905 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a3ae2-41a7-4e1b-86bc-22b987257ff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d66a5d034f04f928c348267d35d0cdf)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:31.320836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a3ae2-41a7-4e1b-86bc-22b987257ff8 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:31.369730 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ae0a3ae2-41a7-4e1b-86bc-22b987257ff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d66a5d034f04f928c348267d35d0cdf'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:31.370365 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ae0a3ae2-41a7-4e1b-86bc-22b987257ff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d66a5d034f04f928c348267d35d0cdf'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:31.371075 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ae0a3ae2-41a7-4e1b-86bc-22b987257ff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d66a5d034f04f928c348267d35d0cdf'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:31.378377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ae0a3ae2-41a7-4e1b-86bc-22b987257ff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d66a5d034f04f928c348267d35d0cdf'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:31.391513 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ae0a3ae2-41a7-4e1b-86bc-22b987257ff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d66a5d034f04f928c348267d35d0cdf'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:31.405537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1406/2819] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:40:31 2022] DELETE /identity/v3/users/0d66a5d034f04f928c348267d35d0cdf => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:31.410585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042894e-bef3-48fc-bbc7-2c629800f06a None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:31.411096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042894e-bef3-48fc-bbc7-2c629800f06a None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:31.411096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042894e-bef3-48fc-bbc7-2c629800f06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:31.432048 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-3042894e-bef3-48fc-bbc7-2c629800f06a None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:31.433253 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3042894e-bef3-48fc-bbc7-2c629800f06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:31.491783 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1414/2820] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:40:31.502487 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a3ae2-41a7-4e1b-86bc-22b987257ff8 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:31.504929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ae0a3ae2-41a7-4e1b-86bc-22b987257ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:31.575793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a3ae2-41a7-4e1b-86bc-22b987257ff8 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:31.582414 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990542cd-a92b-4395-96b6-37a53c4f4ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:31.583337 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990542cd-a92b-4395-96b6-37a53c4f4ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:31.583704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990542cd-a92b-4395-96b6-37a53c4f4ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:31.583952 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990542cd-a92b-4395-96b6-37a53c4f4ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:31.596671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990542cd-a92b-4395-96b6-37a53c4f4ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:31.598111 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990542cd-a92b-4395-96b6-37a53c4f4ee7 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:31.607682 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1407/2821] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:40:31.879776 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042894e-bef3-48fc-bbc7-2c629800f06a None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:31.881293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3042894e-bef3-48fc-bbc7-2c629800f06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:31.893270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042894e-bef3-48fc-bbc7-2c629800f06a None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:31.897913 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:31.898585 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:31.898893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:31.899160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] PATH_INFO: `/v3/projects/987ae45b30094b119441849ca3df098b` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:31.900279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=987ae45b30094b119441849ca3df098b)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:31.902236 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:31.949497 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '987ae45b30094b119441849ca3df098b', 'request_id': 'req-a5a7205b-8d40-4a63-b308-67ea01c955fe'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:31.955694 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '987ae45b30094b119441849ca3df098b', 'request_id': 'req-a5a7205b-8d40-4a63-b308-67ea01c955fe'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:31.961443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '987ae45b30094b119441849ca3df098b', 'request_id': 'req-a5a7205b-8d40-4a63-b308-67ea01c955fe'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:31.976062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] The token cache is being invalidate because project 987ae45b30094b119441849ca3df098b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:40:31.976351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:31.977059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:31.978765 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1415/2822] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:40:31 2022] DELETE /identity/v3/projects/987ae45b30094b119441849ca3df098b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:31.983139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990542cd-a92b-4395-96b6-37a53c4f4ee7 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:31.992939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990542cd-a92b-4395-96b6-37a53c4f4ee7 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:32.011037 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:32.052382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02a058-62a8-42b0-92b8-2e7a60275853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:32.053043 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02a058-62a8-42b0-92b8-2e7a60275853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:32.053322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02a058-62a8-42b0-92b8-2e7a60275853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:32.053594 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02a058-62a8-42b0-92b8-2e7a60275853 admin admin] PATH_INFO: `/v3/users/9774137b6b254da1bbeeddad29a26d84` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:32.054781 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02a058-62a8-42b0-92b8-2e7a60275853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9774137b6b254da1bbeeddad29a26d84)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:32.056699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02a058-62a8-42b0-92b8-2e7a60275853 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:32.105507 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-7b02a058-62a8-42b0-92b8-2e7a60275853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9774137b6b254da1bbeeddad29a26d84'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:32.106155 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-7b02a058-62a8-42b0-92b8-2e7a60275853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9774137b6b254da1bbeeddad29a26d84'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:32.106776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-7b02a058-62a8-42b0-92b8-2e7a60275853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9774137b6b254da1bbeeddad29a26d84'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:32.113467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-7b02a058-62a8-42b0-92b8-2e7a60275853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9774137b6b254da1bbeeddad29a26d84'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:32.125137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-7b02a058-62a8-42b0-92b8-2e7a60275853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9774137b6b254da1bbeeddad29a26d84'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:32.139560 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1408/2823] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:40:31 2022] DELETE /identity/v3/users/9774137b6b254da1bbeeddad29a26d84 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:32.420305 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:32.422492 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:32.436144 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7205b-8d40-4a63-b308-67ea01c955fe admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:32.492276 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326318fb-0de0-4ae4-bbae-166a00fe0b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:32.492904 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326318fb-0de0-4ae4-bbae-166a00fe0b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:32.493180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326318fb-0de0-4ae4-bbae-166a00fe0b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:32.493447 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326318fb-0de0-4ae4-bbae-166a00fe0b45 admin admin] PATH_INFO: `/v3/projects/738fd57500434dfaa46c095427fb71d8` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:32.494653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326318fb-0de0-4ae4-bbae-166a00fe0b45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=738fd57500434dfaa46c095427fb71d8)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:32.496572 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326318fb-0de0-4ae4-bbae-166a00fe0b45 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:32.531115 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:32.552150 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-326318fb-0de0-4ae4-bbae-166a00fe0b45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738fd57500434dfaa46c095427fb71d8', 'request_id': 'req-326318fb-0de0-4ae4-bbae-166a00fe0b45'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:32.558299 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-326318fb-0de0-4ae4-bbae-166a00fe0b45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738fd57500434dfaa46c095427fb71d8', 'request_id': 'req-326318fb-0de0-4ae4-bbae-166a00fe0b45'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:32.564957 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-326318fb-0de0-4ae4-bbae-166a00fe0b45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738fd57500434dfaa46c095427fb71d8', 'request_id': 'req-326318fb-0de0-4ae4-bbae-166a00fe0b45'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:32.579988 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-326318fb-0de0-4ae4-bbae-166a00fe0b45 admin admin] The token cache is being invalidate because project 738fd57500434dfaa46c095427fb71d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:40:32.580275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-326318fb-0de0-4ae4-bbae-166a00fe0b45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:32.580883 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-326318fb-0de0-4ae4-bbae-166a00fe0b45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:32.582406 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1416/2824] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:40:32 2022] DELETE /identity/v3/projects/738fd57500434dfaa46c095427fb71d8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:32.587558 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c864e-c0b1-47f8-882d-8cb1668e094b None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:32.588020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c864e-c0b1-47f8-882d-8cb1668e094b None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:32.588020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c864e-c0b1-47f8-882d-8cb1668e094b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:32.608543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-798c864e-c0b1-47f8-882d-8cb1668e094b None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:32.609756 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-798c864e-c0b1-47f8-882d-8cb1668e094b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:32.670383 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1409/2825] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:32.683844 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060008e5-d670-4116-b32a-66ae8d238c16 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:32.684212 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060008e5-d670-4116-b32a-66ae8d238c16 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:32.684490 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060008e5-d670-4116-b32a-66ae8d238c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:32.709611 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-060008e5-d670-4116-b32a-66ae8d238c16 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:32.710878 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-060008e5-d670-4116-b32a-66ae8d238c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:32.721675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1417/2826] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:32.728358 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c864e-c0b1-47f8-882d-8cb1668e094b None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:32.740533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c864e-c0b1-47f8-882d-8cb1668e094b None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:32.746376 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761f6a0-462a-471b-b03d-7c7a052da79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:32.747012 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761f6a0-462a-471b-b03d-7c7a052da79f admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:32.747285 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761f6a0-462a-471b-b03d-7c7a052da79f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:32.747550 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761f6a0-462a-471b-b03d-7c7a052da79f admin admin] PATH_INFO: `/v3/domains` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:32.748231 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761f6a0-462a-471b-b03d-7c7a052da79f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81510) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:40:32.748597 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761f6a0-462a-471b-b03d-7c7a052da79f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:32.750200 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761f6a0-462a-471b-b03d-7c7a052da79f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:32.762289 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1410/2827] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:40:32.767117 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060008e5-d670-4116-b32a-66ae8d238c16 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:32.780622 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060008e5-d670-4116-b32a-66ae8d238c16 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:32.786064 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb5b51a-ff45-4c4f-ba4f-149b11962478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:32.786685 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5b51a-ff45-4c4f-ba4f-149b11962478 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:32.786960 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5b51a-ff45-4c4f-ba4f-149b11962478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:32.787230 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5b51a-ff45-4c4f-ba4f-149b11962478 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:32.787847 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb5b51a-ff45-4c4f-ba4f-149b11962478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:32.789747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb5b51a-ff45-4c4f-ba4f-149b11962478 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:32.819652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1418/2828] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:40:32 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:40:32.823882 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761f6a0-462a-471b-b03d-7c7a052da79f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:32.834098 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761f6a0-462a-471b-b03d-7c7a052da79f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:32.861801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:32.902327 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea992d2-4737-4dc1-9b15-ea04afe223a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:32.902973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea992d2-4737-4dc1-9b15-ea04afe223a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:32.903245 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea992d2-4737-4dc1-9b15-ea04afe223a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:32.903514 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea992d2-4737-4dc1-9b15-ea04afe223a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:32.904110 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea992d2-4737-4dc1-9b15-ea04afe223a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:32.906127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea992d2-4737-4dc1-9b15-ea04afe223a1 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:32.928464 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1411/2829] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:40:32 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:40:32.932822 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb5b51a-ff45-4c4f-ba4f-149b11962478 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:32.944134 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb5b51a-ff45-4c4f-ba4f-149b11962478 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:32.950011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66ecde-130e-4488-9556-749bf05c3238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:32.950652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66ecde-130e-4488-9556-749bf05c3238 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:32.950929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66ecde-130e-4488-9556-749bf05c3238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:32.951186 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66ecde-130e-4488-9556-749bf05c3238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:32.951850 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66ecde-130e-4488-9556-749bf05c3238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:32.953492 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66ecde-130e-4488-9556-749bf05c3238 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:32.963990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1419/2830] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:32.968268 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea992d2-4737-4dc1-9b15-ea04afe223a1 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:32.978301 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea992d2-4737-4dc1-9b15-ea04afe223a1 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:32.983384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a9972-d97f-465c-9b3d-7f59de663a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:32.983968 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a9972-d97f-465c-9b3d-7f59de663a89 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:32.984244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a9972-d97f-465c-9b3d-7f59de663a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:32.984507 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a9972-d97f-465c-9b3d-7f59de663a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:32.985177 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a9972-d97f-465c-9b3d-7f59de663a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:32.986804 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a9972-d97f-465c-9b3d-7f59de663a89 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:32.996880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1412/2831] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:40:33.001148 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66ecde-130e-4488-9556-749bf05c3238 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:33.010964 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66ecde-130e-4488-9556-749bf05c3238 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:33.015614 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befbd1c-e1f8-4edf-b9aa-8e7e1221b738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:33.016240 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befbd1c-e1f8-4edf-b9aa-8e7e1221b738 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:33.016516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befbd1c-e1f8-4edf-b9aa-8e7e1221b738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:33.016784 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befbd1c-e1f8-4edf-b9aa-8e7e1221b738 admin admin] PATH_INFO: `/v3/projects/e7f2b85aac7f443b85e2e19907febe5b/users/9ec0cd0850c847c7a66c68735e60466c/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:33.031376 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befbd1c-e1f8-4edf-b9aa-8e7e1221b738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f2b85aac7f443b85e2e19907febe5b, user_id=9ec0cd0850c847c7a66c68735e60466c, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:33.033996 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befbd1c-e1f8-4edf-b9aa-8e7e1221b738 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:33.049266 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1420/2832] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:40:33 2022] PUT /identity/v3/projects/e7f2b85aac7f443b85e2e19907febe5b/users/9ec0cd0850c847c7a66c68735e60466c/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:40:33.059602 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6daa9a2-8cab-4d0e-bc1d-2d3b6d58b6e6 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:33.059907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6daa9a2-8cab-4d0e-bc1d-2d3b6d58b6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:33.060171 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6daa9a2-8cab-4d0e-bc1d-2d3b6d58b6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:33.088906 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-a6daa9a2-8cab-4d0e-bc1d-2d3b6d58b6e6 None None] MFA Rules not processed for user `9ec0cd0850c847c7a66c68735e60466c`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:33.090127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-a6daa9a2-8cab-4d0e-bc1d-2d3b6d58b6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:33.156153 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1413/2833] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:40:33.161178 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befbd1c-e1f8-4edf-b9aa-8e7e1221b738 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:33.223727 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befbd1c-e1f8-4edf-b9aa-8e7e1221b738 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:33.229368 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5a9ec-63ae-4e47-a7b9-1874da711fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:33.229969 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a9ec-63ae-4e47-a7b9-1874da711fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:33.234395 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a9ec-63ae-4e47-a7b9-1874da711fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:33.234395 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a9ec-63ae-4e47-a7b9-1874da711fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:33.234395 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5a9ec-63ae-4e47-a7b9-1874da711fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:33.234395 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5a9ec-63ae-4e47-a7b9-1874da711fd2 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:33.266559 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1421/2834] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:40:33 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:40:33.270823 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6daa9a2-8cab-4d0e-bc1d-2d3b6d58b6e6 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:33.282043 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6daa9a2-8cab-4d0e-bc1d-2d3b6d58b6e6 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:33.345758 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecf52c-150e-4b86-9f00-8bbb2549cc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:33.346373 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecf52c-150e-4b86-9f00-8bbb2549cc98 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:33.346678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecf52c-150e-4b86-9f00-8bbb2549cc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:33.346951 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecf52c-150e-4b86-9f00-8bbb2549cc98 admin admin] PATH_INFO: `/v3/users` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:33.347593 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecf52c-150e-4b86-9f00-8bbb2549cc98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:33.349496 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecf52c-150e-4b86-9f00-8bbb2549cc98 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:33.370464 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1414/2835] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:40:33 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:40:33.374657 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5a9ec-63ae-4e47-a7b9-1874da711fd2 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:33.384903 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5a9ec-63ae-4e47-a7b9-1874da711fd2 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:33.389747 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6181603-ee65-4076-93f8-8ffc9e095ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:33.390345 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6181603-ee65-4076-93f8-8ffc9e095ebe admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:33.390655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6181603-ee65-4076-93f8-8ffc9e095ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:33.390929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6181603-ee65-4076-93f8-8ffc9e095ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:33.391571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6181603-ee65-4076-93f8-8ffc9e095ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:33.393172 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6181603-ee65-4076-93f8-8ffc9e095ebe admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:33.403683 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1422/2836] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:33.407869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecf52c-150e-4b86-9f00-8bbb2549cc98 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:33.418119 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecf52c-150e-4b86-9f00-8bbb2549cc98 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:33.423413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7df12-eb39-4b4e-b2e0-ae929309093a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:33.424064 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7df12-eb39-4b4e-b2e0-ae929309093a admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:33.424339 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7df12-eb39-4b4e-b2e0-ae929309093a admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:33.424606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7df12-eb39-4b4e-b2e0-ae929309093a admin admin] PATH_INFO: `/v3/projects/4c28b65d7a4e4ba9a5e68daf6416d6c9/users/64414096ad4b4129b4e7ae366357a832/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:33.439740 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7df12-eb39-4b4e-b2e0-ae929309093a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c28b65d7a4e4ba9a5e68daf6416d6c9, user_id=64414096ad4b4129b4e7ae366357a832, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:33.442801 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7df12-eb39-4b4e-b2e0-ae929309093a admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:33.457962 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1415/2837] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:40:33 2022] PUT /identity/v3/projects/4c28b65d7a4e4ba9a5e68daf6416d6c9/users/64414096ad4b4129b4e7ae366357a832/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:33.462174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6181603-ee65-4076-93f8-8ffc9e095ebe admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:33.472549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6181603-ee65-4076-93f8-8ffc9e095ebe admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:33.493378 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:33.533653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2f3ba-9806-4fc7-a319-5061403a8fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:33.534269 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2f3ba-9806-4fc7-a319-5061403a8fad admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:33.534627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2f3ba-9806-4fc7-a319-5061403a8fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:33.535250 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2f3ba-9806-4fc7-a319-5061403a8fad admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:33.535924 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2f3ba-9806-4fc7-a319-5061403a8fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:33.537546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2f3ba-9806-4fc7-a319-5061403a8fad admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:33.548770 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1423/2838] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:33.553116 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7df12-eb39-4b4e-b2e0-ae929309093a admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:33.558824 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:33.566003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7df12-eb39-4b4e-b2e0-ae929309093a admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:33.572546 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577dd34-ceac-46d5-92bf-dcdf653be961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:33.573292 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577dd34-ceac-46d5-92bf-dcdf653be961 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:33.573566 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577dd34-ceac-46d5-92bf-dcdf653be961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:33.573833 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577dd34-ceac-46d5-92bf-dcdf653be961 admin admin] PATH_INFO: `/v3/projects/4c28b65d7a4e4ba9a5e68daf6416d6c9/users/64414096ad4b4129b4e7ae366357a832/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:33.575738 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577dd34-ceac-46d5-92bf-dcdf653be961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c28b65d7a4e4ba9a5e68daf6416d6c9, user_id=64414096ad4b4129b4e7ae366357a832, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:33.578386 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577dd34-ceac-46d5-92bf-dcdf653be961 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:33.588968 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1416/2839] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:40:33 2022] PUT /identity/v3/projects/4c28b65d7a4e4ba9a5e68daf6416d6c9/users/64414096ad4b4129b4e7ae366357a832/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:40:33.599248 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfcb2b-2327-4f68-8787-8735d9c8cd32 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:33.599533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfcb2b-2327-4f68-8787-8735d9c8cd32 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:33.599802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfcb2b-2327-4f68-8787-8735d9c8cd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:33.628486 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-89dfcb2b-2327-4f68-8787-8735d9c8cd32 None None] MFA Rules not processed for user `64414096ad4b4129b4e7ae366357a832`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:33.629702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-89dfcb2b-2327-4f68-8787-8735d9c8cd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:33.703820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1424/2840] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:33.715326 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577dd34-ceac-46d5-92bf-dcdf653be961 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:33.717217 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-9577dd34-ceac-46d5-92bf-dcdf653be961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:33.776284 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577dd34-ceac-46d5-92bf-dcdf653be961 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:33.781872 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c155f-0db4-4168-a447-b1e86f4e21bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:33.782489 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c155f-0db4-4168-a447-b1e86f4e21bf service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:33.782798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c155f-0db4-4168-a447-b1e86f4e21bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:33.783132 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c155f-0db4-4168-a447-b1e86f4e21bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:33.794081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c155f-0db4-4168-a447-b1e86f4e21bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:33.795569 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c155f-0db4-4168-a447-b1e86f4e21bf service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:33.805378 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1417/2841] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:40:33.853702 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfcb2b-2327-4f68-8787-8735d9c8cd32 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:33.855340 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-89dfcb2b-2327-4f68-8787-8735d9c8cd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:33.868751 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfcb2b-2327-4f68-8787-8735d9c8cd32 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:33.872696 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4e105-4ee6-4c92-81c8-063d2004c540 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:33.873298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4e105-4ee6-4c92-81c8-063d2004c540 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:33.873626 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4e105-4ee6-4c92-81c8-063d2004c540 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:33.873897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4e105-4ee6-4c92-81c8-063d2004c540 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:33.936412 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4e105-4ee6-4c92-81c8-063d2004c540 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:33.937843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4e105-4ee6-4c92-81c8-063d2004c540 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:33.946897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1425/2842] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:40:37.381777 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c155f-0db4-4168-a447-b1e86f4e21bf service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:37.384259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ff6c155f-0db4-4168-a447-b1e86f4e21bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:37.455215 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c155f-0db4-4168-a447-b1e86f4e21bf service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:37.459329 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577262d-c691-41b5-a046-8a3336e894a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:37.461235 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4e105-4ee6-4c92-81c8-063d2004c540 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:37.462564 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577262d-c691-41b5-a046-8a3336e894a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:37.462918 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577262d-c691-41b5-a046-8a3336e894a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:37.463203 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577262d-c691-41b5-a046-8a3336e894a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:37.465207 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-6577262d-c691-41b5-a046-8a3336e894a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:37.511571 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:37.527305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577262d-c691-41b5-a046-8a3336e894a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:37.528751 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577262d-c691-41b5-a046-8a3336e894a9 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:37.529977 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4e105-4ee6-4c92-81c8-063d2004c540 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:37.535270 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483de4e-9d0a-464b-b381-db8399023b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:37.535895 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483de4e-9d0a-464b-b381-db8399023b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:37.536170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483de4e-9d0a-464b-b381-db8399023b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:37.536442 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483de4e-9d0a-464b-b381-db8399023b2b admin admin] PATH_INFO: `/v3/users/9ec0cd0850c847c7a66c68735e60466c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:37.537536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3483de4e-9d0a-464b-b381-db8399023b2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec0cd0850c847c7a66c68735e60466c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:37.538467 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1418/2843] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:40:37.539545 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3483de4e-9d0a-464b-b381-db8399023b2b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:37.589759 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3483de4e-9d0a-464b-b381-db8399023b2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec0cd0850c847c7a66c68735e60466c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:37.590412 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3483de4e-9d0a-464b-b381-db8399023b2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec0cd0850c847c7a66c68735e60466c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:37.591075 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3483de4e-9d0a-464b-b381-db8399023b2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec0cd0850c847c7a66c68735e60466c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:37.603290 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3483de4e-9d0a-464b-b381-db8399023b2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec0cd0850c847c7a66c68735e60466c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:37.615863 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-3483de4e-9d0a-464b-b381-db8399023b2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec0cd0850c847c7a66c68735e60466c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:37.634579 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1426/2844] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:40:37 2022] DELETE /identity/v3/users/9ec0cd0850c847c7a66c68735e60466c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:40:37.638868 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5273e2-1609-4859-bb6c-7b0ac6339c09 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:37.640046 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5273e2-1609-4859-bb6c-7b0ac6339c09 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:37.640471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5273e2-1609-4859-bb6c-7b0ac6339c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:37.660458 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-cd5273e2-1609-4859-bb6c-7b0ac6339c09 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:37.661674 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cd5273e2-1609-4859-bb6c-7b0ac6339c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:37.710668 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:37.728816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1419/2845] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:40:37.739860 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483de4e-9d0a-464b-b381-db8399023b2b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:37.741348 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-3483de4e-9d0a-464b-b381-db8399023b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:37.801491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483de4e-9d0a-464b-b381-db8399023b2b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:37.805366 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2da03-c2dd-4bc2-a63c-68a59078255c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:37.805983 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2da03-c2dd-4bc2-a63c-68a59078255c service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:37.806262 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2da03-c2dd-4bc2-a63c-68a59078255c service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:37.806550 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2da03-c2dd-4bc2-a63c-68a59078255c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:37.818191 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2da03-c2dd-4bc2-a63c-68a59078255c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:37.819779 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2da03-c2dd-4bc2-a63c-68a59078255c service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:37.830630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1427/2846] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:40:38.118416 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5273e2-1609-4859-bb6c-7b0ac6339c09 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:38.120004 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-cd5273e2-1609-4859-bb6c-7b0ac6339c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:38.152776 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5273e2-1609-4859-bb6c-7b0ac6339c09 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:38.159096 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:38.160329 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:38.160678 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:38.161014 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] PATH_INFO: `/v3/projects/e7f2b85aac7f443b85e2e19907febe5b` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:38.163198 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7f2b85aac7f443b85e2e19907febe5b)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:38.166006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:38.214593 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7f2b85aac7f443b85e2e19907febe5b', 'request_id': 'req-ce670eac-4a09-437c-8ae9-f9a651e8acb6'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.221062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7f2b85aac7f443b85e2e19907febe5b', 'request_id': 'req-ce670eac-4a09-437c-8ae9-f9a651e8acb6'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.229692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7f2b85aac7f443b85e2e19907febe5b', 'request_id': 'req-ce670eac-4a09-437c-8ae9-f9a651e8acb6'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.240290 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] The token cache is being invalidate because project e7f2b85aac7f443b85e2e19907febe5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:40:38.240574 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.241230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.242822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1420/2847] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:40:38 2022] DELETE /identity/v3/projects/e7f2b85aac7f443b85e2e19907febe5b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:38.247227 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2da03-c2dd-4bc2-a63c-68a59078255c service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:38.256968 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2da03-c2dd-4bc2-a63c-68a59078255c service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:38.315093 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fdf4f-39a5-45bf-8816-275b1b0d9f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:38.315831 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fdf4f-39a5-45bf-8816-275b1b0d9f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:38.316102 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fdf4f-39a5-45bf-8816-275b1b0d9f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:38.316373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fdf4f-39a5-45bf-8816-275b1b0d9f92 admin admin] PATH_INFO: `/v3/users/64414096ad4b4129b4e7ae366357a832` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:38.318189 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9fdf4f-39a5-45bf-8816-275b1b0d9f92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64414096ad4b4129b4e7ae366357a832)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:38.320182 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9fdf4f-39a5-45bf-8816-275b1b0d9f92 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:38.326300 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:38.371925 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1e9fdf4f-39a5-45bf-8816-275b1b0d9f92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64414096ad4b4129b4e7ae366357a832'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.372751 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1e9fdf4f-39a5-45bf-8816-275b1b0d9f92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64414096ad4b4129b4e7ae366357a832'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.373444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1e9fdf4f-39a5-45bf-8816-275b1b0d9f92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64414096ad4b4129b4e7ae366357a832'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.380732 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1e9fdf4f-39a5-45bf-8816-275b1b0d9f92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64414096ad4b4129b4e7ae366357a832'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.392750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-1e9fdf4f-39a5-45bf-8816-275b1b0d9f92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64414096ad4b4129b4e7ae366357a832'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.406070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1428/2848] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:40:38 2022] DELETE /identity/v3/users/64414096ad4b4129b4e7ae366357a832 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:38.708911 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:38.710427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:38.726013 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce670eac-4a09-437c-8ae9-f9a651e8acb6 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:38.787265 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0349104-9b8a-49de-8af4-e11bc641d582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:38.788047 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0349104-9b8a-49de-8af4-e11bc641d582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:38.788356 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0349104-9b8a-49de-8af4-e11bc641d582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:38.788632 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0349104-9b8a-49de-8af4-e11bc641d582 admin admin] PATH_INFO: `/v3/projects/4c28b65d7a4e4ba9a5e68daf6416d6c9` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:38.789854 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0349104-9b8a-49de-8af4-e11bc641d582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c28b65d7a4e4ba9a5e68daf6416d6c9)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:38.792006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0349104-9b8a-49de-8af4-e11bc641d582 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:38.846100 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d0349104-9b8a-49de-8af4-e11bc641d582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c28b65d7a4e4ba9a5e68daf6416d6c9', 'request_id': 'req-d0349104-9b8a-49de-8af4-e11bc641d582'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.852429 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d0349104-9b8a-49de-8af4-e11bc641d582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c28b65d7a4e4ba9a5e68daf6416d6c9', 'request_id': 'req-d0349104-9b8a-49de-8af4-e11bc641d582'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.861476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d0349104-9b8a-49de-8af4-e11bc641d582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c28b65d7a4e4ba9a5e68daf6416d6c9', 'request_id': 'req-d0349104-9b8a-49de-8af4-e11bc641d582'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.869499 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:38.873875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d0349104-9b8a-49de-8af4-e11bc641d582 admin admin] The token cache is being invalidate because project 4c28b65d7a4e4ba9a5e68daf6416d6c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:40:38.874336 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d0349104-9b8a-49de-8af4-e11bc641d582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.875182 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d0349104-9b8a-49de-8af4-e11bc641d582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:38.877523 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1421/2849] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:40:38 2022] DELETE /identity/v3/projects/4c28b65d7a4e4ba9a5e68daf6416d6c9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:38.882703 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc47d6-4aeb-43e0-b16c-fee3310a411d None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:38.883169 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc47d6-4aeb-43e0-b16c-fee3310a411d None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:38.883351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc47d6-4aeb-43e0-b16c-fee3310a411d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:38.904173 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-fdfc47d6-4aeb-43e0-b16c-fee3310a411d None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:38.905420 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-fdfc47d6-4aeb-43e0-b16c-fee3310a411d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:38.968672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1429/2850] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:38.977468 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfdc34-79ea-4284-a7ea-49a78856dbee None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:38.977927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfdc34-79ea-4284-a7ea-49a78856dbee None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:38.978209 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfdc34-79ea-4284-a7ea-49a78856dbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:39.002006 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-debfdc34-79ea-4284-a7ea-49a78856dbee None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:39.003478 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-debfdc34-79ea-4284-a7ea-49a78856dbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:39.012957 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1422/2851] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:40:39.017546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc47d6-4aeb-43e0-b16c-fee3310a411d None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:39.029406 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc47d6-4aeb-43e0-b16c-fee3310a411d None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:39.035065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21965e-5f3d-4f11-a1b5-b396ed2996f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:39.035675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21965e-5f3d-4f11-a1b5-b396ed2996f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:39.035950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21965e-5f3d-4f11-a1b5-b396ed2996f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:39.036223 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21965e-5f3d-4f11-a1b5-b396ed2996f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:39.036869 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21965e-5f3d-4f11-a1b5-b396ed2996f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:40:39.037170 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21965e-5f3d-4f11-a1b5-b396ed2996f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:39.038800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21965e-5f3d-4f11-a1b5-b396ed2996f7 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:39.051444 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1430/2852] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:40:39.055991 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfdc34-79ea-4284-a7ea-49a78856dbee None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:39.065802 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfdc34-79ea-4284-a7ea-49a78856dbee None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:39.071092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7bd82-8218-4b17-93d1-159910cf882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:39.071698 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7bd82-8218-4b17-93d1-159910cf882c admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:39.071977 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7bd82-8218-4b17-93d1-159910cf882c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:39.072239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7bd82-8218-4b17-93d1-159910cf882c admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:39.072871 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7bd82-8218-4b17-93d1-159910cf882c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:39.074889 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7bd82-8218-4b17-93d1-159910cf882c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:39.104404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1423/2853] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:40:39 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 16:40:39.109092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21965e-5f3d-4f11-a1b5-b396ed2996f7 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:39.119347 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21965e-5f3d-4f11-a1b5-b396ed2996f7 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:39.177846 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5b43f-b4c3-4edb-8322-bd15192a276e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:39.178387 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5b43f-b4c3-4edb-8322-bd15192a276e admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:39.178883 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5b43f-b4c3-4edb-8322-bd15192a276e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:39.179167 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5b43f-b4c3-4edb-8322-bd15192a276e admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:39.179818 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5b43f-b4c3-4edb-8322-bd15192a276e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:39.181813 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5b43f-b4c3-4edb-8322-bd15192a276e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:39.203245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1431/2854] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:40:39 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:40:39.208085 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7bd82-8218-4b17-93d1-159910cf882c admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:39.218969 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7bd82-8218-4b17-93d1-159910cf882c admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:39.224671 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f06f56-2cbf-4070-9896-44319334fc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:39.225277 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f06f56-2cbf-4070-9896-44319334fc90 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:39.225556 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f06f56-2cbf-4070-9896-44319334fc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:39.225953 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f06f56-2cbf-4070-9896-44319334fc90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:39.226635 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f06f56-2cbf-4070-9896-44319334fc90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:39.228230 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f06f56-2cbf-4070-9896-44319334fc90 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:39.238994 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1424/2855] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:40:39.243639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5b43f-b4c3-4edb-8322-bd15192a276e admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:39.249932 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:39.257436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5b43f-b4c3-4edb-8322-bd15192a276e admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:39.265064 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285b183-5c82-496d-adc7-16d5c09c57eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:39.266972 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285b183-5c82-496d-adc7-16d5c09c57eb admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:39.267523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285b183-5c82-496d-adc7-16d5c09c57eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:39.268062 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285b183-5c82-496d-adc7-16d5c09c57eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:39.269373 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285b183-5c82-496d-adc7-16d5c09c57eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:39.271177 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285b183-5c82-496d-adc7-16d5c09c57eb admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:39.281538 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1432/2856] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:39.286305 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f06f56-2cbf-4070-9896-44319334fc90 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:39.304090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f06f56-2cbf-4070-9896-44319334fc90 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:39.310131 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f4036-2a42-4151-a1c8-76c38e048c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:39.310843 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f4036-2a42-4151-a1c8-76c38e048c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:39.311118 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f4036-2a42-4151-a1c8-76c38e048c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:39.311407 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f4036-2a42-4151-a1c8-76c38e048c32 admin admin] PATH_INFO: `/v3/projects/fd68e82b15a148b382fed0455e767628/users/4a4b858db3ae40fe9a64bc447d2f589c/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:39.316791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285b183-5c82-496d-adc7-16d5c09c57eb admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:39.318556 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b285b183-5c82-496d-adc7-16d5c09c57eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:39.327818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f4036-2a42-4151-a1c8-76c38e048c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd68e82b15a148b382fed0455e767628, user_id=4a4b858db3ae40fe9a64bc447d2f589c, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:39.330490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f4036-2a42-4151-a1c8-76c38e048c32 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:39.344929 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1425/2857] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:40:39 2022] PUT /identity/v3/projects/fd68e82b15a148b382fed0455e767628/users/4a4b858db3ae40fe9a64bc447d2f589c/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:40:39.350084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f450b-2802-4e9f-9f14-f6fe7acf5262 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:39.350341 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f450b-2802-4e9f-9f14-f6fe7acf5262 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:39.350692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f450b-2802-4e9f-9f14-f6fe7acf5262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:39.369237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-448f450b-2802-4e9f-9f14-f6fe7acf5262 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:39.370476 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-448f450b-2802-4e9f-9f14-f6fe7acf5262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:39.379569 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285b183-5c82-496d-adc7-16d5c09c57eb admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:39.428132 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7de1a-2328-4b40-9e92-4a5514aed047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:39.428649 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7de1a-2328-4b40-9e92-4a5514aed047 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:39.428823 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1426/2858] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:40:39.428946 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7de1a-2328-4b40-9e92-4a5514aed047 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:39.430879 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7de1a-2328-4b40-9e92-4a5514aed047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:39.432934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-4da7de1a-2328-4b40-9e92-4a5514aed047 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:39.439311 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f450b-2802-4e9f-9f14-f6fe7acf5262 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:39.445929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7de1a-2328-4b40-9e92-4a5514aed047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:39.447419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7de1a-2328-4b40-9e92-4a5514aed047 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:39.449062 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f450b-2802-4e9f-9f14-f6fe7acf5262 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:39.452869 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebabfa8-4b66-4b42-b5bb-e5f8175abffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:39.453525 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebabfa8-4b66-4b42-b5bb-e5f8175abffa service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:39.453808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebabfa8-4b66-4b42-b5bb-e5f8175abffa service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:39.454082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebabfa8-4b66-4b42-b5bb-e5f8175abffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:39.457232 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1433/2859] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:40:39.463701 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebabfa8-4b66-4b42-b5bb-e5f8175abffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:39.465149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebabfa8-4b66-4b42-b5bb-e5f8175abffa service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:39.475403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1427/2860] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:40:40.453889 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a046b1-2d8f-4a4f-b23a-3a5f8940c912 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:40.454428 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a046b1-2d8f-4a4f-b23a-3a5f8940c912 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:40.454428 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a046b1-2d8f-4a4f-b23a-3a5f8940c912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:40.483279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a9a046b1-2d8f-4a4f-b23a-3a5f8940c912 None None] MFA Rules not processed for user `4a4b858db3ae40fe9a64bc447d2f589c`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:40.484536 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a9a046b1-2d8f-4a4f-b23a-3a5f8940c912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:40.562153 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1434/2861] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:40:40.567094 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebabfa8-4b66-4b42-b5bb-e5f8175abffa service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:40.580771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebabfa8-4b66-4b42-b5bb-e5f8175abffa service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:40.585704 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17435877-e1dd-41ec-99c1-392e0f8ade23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:40.586404 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17435877-e1dd-41ec-99c1-392e0f8ade23 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:40.586803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17435877-e1dd-41ec-99c1-392e0f8ade23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:40.587147 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17435877-e1dd-41ec-99c1-392e0f8ade23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:40.587850 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17435877-e1dd-41ec-99c1-392e0f8ade23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:40.589893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17435877-e1dd-41ec-99c1-392e0f8ade23 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:40.633473 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1428/2862] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:40:40 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:40:40.635071 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a046b1-2d8f-4a4f-b23a-3a5f8940c912 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:40.649982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a046b1-2d8f-4a4f-b23a-3a5f8940c912 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:40.704781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=100 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:40.730736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1cf9a-c62a-4fdc-8737-72f462621ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:40.731364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1cf9a-c62a-4fdc-8737-72f462621ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:40.731638 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1cf9a-c62a-4fdc-8737-72f462621ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:40.731903 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1cf9a-c62a-4fdc-8737-72f462621ea3 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:40.732553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1cf9a-c62a-4fdc-8737-72f462621ea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:40.734583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1cf9a-c62a-4fdc-8737-72f462621ea3 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:40.758992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1435/2863] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:40:40 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:40:40.763021 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17435877-e1dd-41ec-99c1-392e0f8ade23 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:40.774221 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17435877-e1dd-41ec-99c1-392e0f8ade23 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:40.778712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13852fab-e549-4ac7-a9ef-5a5f911e8318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:40.779319 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13852fab-e549-4ac7-a9ef-5a5f911e8318 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:40.779605 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13852fab-e549-4ac7-a9ef-5a5f911e8318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:40.779893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13852fab-e549-4ac7-a9ef-5a5f911e8318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:40.780533 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13852fab-e549-4ac7-a9ef-5a5f911e8318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:40.782154 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13852fab-e549-4ac7-a9ef-5a5f911e8318 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:40.798439 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1429/2864] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:40.799817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1cf9a-c62a-4fdc-8737-72f462621ea3 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:40.809836 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1cf9a-c62a-4fdc-8737-72f462621ea3 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:40.814485 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e32e0-8bc5-4841-bba9-c3174a572e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:40.815189 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e32e0-8bc5-4841-bba9-c3174a572e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:40.815451 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e32e0-8bc5-4841-bba9-c3174a572e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:40.815713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e32e0-8bc5-4841-bba9-c3174a572e74 admin admin] PATH_INFO: `/v3/projects/3877ced20e18404d81a4958e966d8bfb/users/331bd879dc7949978d466301fe3f44f4/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:40.831604 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e32e0-8bc5-4841-bba9-c3174a572e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3877ced20e18404d81a4958e966d8bfb, user_id=331bd879dc7949978d466301fe3f44f4, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:40.834324 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e32e0-8bc5-4841-bba9-c3174a572e74 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:40.849063 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1436/2865] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:40:40 2022] PUT /identity/v3/projects/3877ced20e18404d81a4958e966d8bfb/users/331bd879dc7949978d466301fe3f44f4/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:40:40.853219 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13852fab-e549-4ac7-a9ef-5a5f911e8318 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:40.864504 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13852fab-e549-4ac7-a9ef-5a5f911e8318 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:40.912551 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:40.924696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ecb9d-caa7-4c65-90ee-ec096009afa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:40.925309 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ecb9d-caa7-4c65-90ee-ec096009afa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:40.925571 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ecb9d-caa7-4c65-90ee-ec096009afa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:40.925836 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ecb9d-caa7-4c65-90ee-ec096009afa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:40.926470 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ecb9d-caa7-4c65-90ee-ec096009afa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:40.928087 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ecb9d-caa7-4c65-90ee-ec096009afa2 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:40.937960 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1430/2866] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 16:40:40.942327 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e32e0-8bc5-4841-bba9-c3174a572e74 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:40.952418 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e32e0-8bc5-4841-bba9-c3174a572e74 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:40.957141 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20bfe6-c249-47db-a432-2495d28ff839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:40.957801 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20bfe6-c249-47db-a432-2495d28ff839 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:40.958076 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20bfe6-c249-47db-a432-2495d28ff839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:40.958340 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20bfe6-c249-47db-a432-2495d28ff839 admin admin] PATH_INFO: `/v3/projects/3877ced20e18404d81a4958e966d8bfb/users/331bd879dc7949978d466301fe3f44f4/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:40.961076 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20bfe6-c249-47db-a432-2495d28ff839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3877ced20e18404d81a4958e966d8bfb, user_id=331bd879dc7949978d466301fe3f44f4, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:40.963298 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20bfe6-c249-47db-a432-2495d28ff839 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:40.974561 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1437/2867] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:40:40 2022] PUT /identity/v3/projects/3877ced20e18404d81a4958e966d8bfb/users/331bd879dc7949978d466301fe3f44f4/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:41.781117 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3719d0-41de-4cbd-9187-eb256f80f7cb None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:41.781728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3719d0-41de-4cbd-9187-eb256f80f7cb None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:41.781728 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3719d0-41de-4cbd-9187-eb256f80f7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:41.810269 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-5e3719d0-41de-4cbd-9187-eb256f80f7cb None None] MFA Rules not processed for user `331bd879dc7949978d466301fe3f44f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:41.811633 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-5e3719d0-41de-4cbd-9187-eb256f80f7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:41.916157 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1431/2868] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:41.928462 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20bfe6-c249-47db-a432-2495d28ff839 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:41.930275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-0e20bfe6-c249-47db-a432-2495d28ff839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:41.992915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20bfe6-c249-47db-a432-2495d28ff839 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:41.996542 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf379b-87c0-4adb-980a-34f3568e3176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:41.997218 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf379b-87c0-4adb-980a-34f3568e3176 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:41.997553 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf379b-87c0-4adb-980a-34f3568e3176 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:41.997905 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf379b-87c0-4adb-980a-34f3568e3176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:42.057217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf379b-87c0-4adb-980a-34f3568e3176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:42.058826 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf379b-87c0-4adb-980a-34f3568e3176 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:42.068445 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1438/2869] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:40:43.569283 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3719d0-41de-4cbd-9187-eb256f80f7cb None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:43.580560 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3719d0-41de-4cbd-9187-eb256f80f7cb None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:43.585252 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baf4fc-7d73-4d8d-a701-22f95a034619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:43.586392 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baf4fc-7d73-4d8d-a701-22f95a034619 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:43.586909 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baf4fc-7d73-4d8d-a701-22f95a034619 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:43.587415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baf4fc-7d73-4d8d-a701-22f95a034619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:43.600282 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86baf4fc-7d73-4d8d-a701-22f95a034619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:43.602068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86baf4fc-7d73-4d8d-a701-22f95a034619 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:43.613248 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1432/2870] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:40:45.316568 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf379b-87c0-4adb-980a-34f3568e3176 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:45.318066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6adf379b-87c0-4adb-980a-34f3568e3176 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:45.378764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:45.398380 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf379b-87c0-4adb-980a-34f3568e3176 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:45.402975 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a5ee8-488c-4e2f-b26b-a857e6876072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:45.403601 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a5ee8-488c-4e2f-b26b-a857e6876072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:45.403880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a5ee8-488c-4e2f-b26b-a857e6876072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:45.404158 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a5ee8-488c-4e2f-b26b-a857e6876072 admin admin] PATH_INFO: `/v3/users/5ae302ea931a42b5938884e79c2ae381` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:45.405336 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a5ee8-488c-4e2f-b26b-a857e6876072 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ae302ea931a42b5938884e79c2ae381)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:45.407357 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a5ee8-488c-4e2f-b26b-a857e6876072 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:45.465472 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-059a5ee8-488c-4e2f-b26b-a857e6876072 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ae302ea931a42b5938884e79c2ae381'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:45.466116 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-059a5ee8-488c-4e2f-b26b-a857e6876072 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ae302ea931a42b5938884e79c2ae381'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:45.466981 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-059a5ee8-488c-4e2f-b26b-a857e6876072 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ae302ea931a42b5938884e79c2ae381'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:45.474603 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-059a5ee8-488c-4e2f-b26b-a857e6876072 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ae302ea931a42b5938884e79c2ae381'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:45.488277 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-059a5ee8-488c-4e2f-b26b-a857e6876072 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ae302ea931a42b5938884e79c2ae381'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:45.503527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1439/2871] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:40:45 2022] DELETE /identity/v3/users/5ae302ea931a42b5938884e79c2ae381 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:45.690913 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baf4fc-7d73-4d8d-a701-22f95a034619 service cinder] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:45.709019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baf4fc-7d73-4d8d-a701-22f95a034619 service cinder] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:45.790806 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:45.791644 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:45.791931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:45.792203 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] PATH_INFO: `/v3/projects/1c6c1e895b5a49009d67f5d7e0ab6058` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:45.794120 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c6c1e895b5a49009d67f5d7e0ab6058)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:45.796334 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:45.847735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c6c1e895b5a49009d67f5d7e0ab6058', 'request_id': 'req-148abf8d-eec6-4cec-a94b-44ff65aa4623'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:45.855323 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c6c1e895b5a49009d67f5d7e0ab6058', 'request_id': 'req-148abf8d-eec6-4cec-a94b-44ff65aa4623'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:45.867022 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c6c1e895b5a49009d67f5d7e0ab6058', 'request_id': 'req-148abf8d-eec6-4cec-a94b-44ff65aa4623'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:45.876173 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:45.891204 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] The token cache is being invalidate because project 1c6c1e895b5a49009d67f5d7e0ab6058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:40:45.891712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:45.892708 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:45.894335 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1433/2872] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:40:45 2022] DELETE /identity/v3/projects/1c6c1e895b5a49009d67f5d7e0ab6058 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:46.345451 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a5ee8-488c-4e2f-b26b-a857e6876072 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:46.346965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-059a5ee8-488c-4e2f-b26b-a857e6876072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:46.366998 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a5ee8-488c-4e2f-b26b-a857e6876072 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:46.431742 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3308c0-ff5e-41c3-8fbf-00ca3dec1d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:46.432387 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3308c0-ff5e-41c3-8fbf-00ca3dec1d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:46.432665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3308c0-ff5e-41c3-8fbf-00ca3dec1d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:46.432936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3308c0-ff5e-41c3-8fbf-00ca3dec1d6f admin admin] PATH_INFO: `/v3/users/208daa6413d34e4496e278fb3fc251a5` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:46.434069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3308c0-ff5e-41c3-8fbf-00ca3dec1d6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=208daa6413d34e4496e278fb3fc251a5)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:46.436160 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3308c0-ff5e-41c3-8fbf-00ca3dec1d6f admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:46.489631 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6c3308c0-ff5e-41c3-8fbf-00ca3dec1d6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '208daa6413d34e4496e278fb3fc251a5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:46.490273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6c3308c0-ff5e-41c3-8fbf-00ca3dec1d6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '208daa6413d34e4496e278fb3fc251a5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:46.490874 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6c3308c0-ff5e-41c3-8fbf-00ca3dec1d6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '208daa6413d34e4496e278fb3fc251a5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:46.497821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6c3308c0-ff5e-41c3-8fbf-00ca3dec1d6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '208daa6413d34e4496e278fb3fc251a5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:46.506151 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:46.511825 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-6c3308c0-ff5e-41c3-8fbf-00ca3dec1d6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '208daa6413d34e4496e278fb3fc251a5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:46.528011 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1440/2873] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:40:46 2022] DELETE /identity/v3/users/208daa6413d34e4496e278fb3fc251a5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:46.709656 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:46.711338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:46.725403 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148abf8d-eec6-4cec-a94b-44ff65aa4623 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:46.790784 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:46.791421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:46.791695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:46.791965 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] PATH_INFO: `/v3/projects/1ba803ac5e71467f853215e19a7f8867` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:46.793082 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ba803ac5e71467f853215e19a7f8867)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:46.795150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:46.843907 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba803ac5e71467f853215e19a7f8867', 'request_id': 'req-18e045d8-a541-46df-aee5-6cc468313109'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:46.849980 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba803ac5e71467f853215e19a7f8867', 'request_id': 'req-18e045d8-a541-46df-aee5-6cc468313109'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:46.860084 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba803ac5e71467f853215e19a7f8867', 'request_id': 'req-18e045d8-a541-46df-aee5-6cc468313109'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:46.870410 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] The token cache is being invalidate because project 1ba803ac5e71467f853215e19a7f8867 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:40:46.870730 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:46.871332 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:46.872803 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1434/2874] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:40:46 2022] DELETE /identity/v3/projects/1ba803ac5e71467f853215e19a7f8867 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:46.876929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3308c0-ff5e-41c3-8fbf-00ca3dec1d6f admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:46.878388 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6c3308c0-ff5e-41c3-8fbf-00ca3dec1d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:46.891923 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3308c0-ff5e-41c3-8fbf-00ca3dec1d6f admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:46.952679 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9e2a7-6598-419b-a093-b5d53c5e4cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:46.953357 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9e2a7-6598-419b-a093-b5d53c5e4cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:46.953643 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9e2a7-6598-419b-a093-b5d53c5e4cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:46.953936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9e2a7-6598-419b-a093-b5d53c5e4cc0 admin admin] PATH_INFO: `/v3/users/b1361af4674349998ad90ce0e79391c5` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:46.955194 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9e2a7-6598-419b-a093-b5d53c5e4cc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1361af4674349998ad90ce0e79391c5)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:46.957200 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9e2a7-6598-419b-a093-b5d53c5e4cc0 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:47.005121 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-93e9e2a7-6598-419b-a093-b5d53c5e4cc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1361af4674349998ad90ce0e79391c5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:47.005672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-93e9e2a7-6598-419b-a093-b5d53c5e4cc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1361af4674349998ad90ce0e79391c5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:47.006279 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-93e9e2a7-6598-419b-a093-b5d53c5e4cc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1361af4674349998ad90ce0e79391c5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:47.013241 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-93e9e2a7-6598-419b-a093-b5d53c5e4cc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1361af4674349998ad90ce0e79391c5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:47.026072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-93e9e2a7-6598-419b-a093-b5d53c5e4cc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1361af4674349998ad90ce0e79391c5'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:47.041383 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1441/2875] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:40:46 2022] DELETE /identity/v3/users/b1361af4674349998ad90ce0e79391c5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:47.225025 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:47.226610 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:47.239997 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e045d8-a541-46df-aee5-6cc468313109 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:47.308504 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:47.309150 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:47.309425 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:47.309694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] PATH_INFO: `/v3/projects/e19301f6dc8c4ed1815a8ab00125c8e5` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:47.310887 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e19301f6dc8c4ed1815a8ab00125c8e5)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:47.312832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:47.337338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:47.361773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e19301f6dc8c4ed1815a8ab00125c8e5', 'request_id': 'req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:47.367841 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e19301f6dc8c4ed1815a8ab00125c8e5', 'request_id': 'req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:47.376982 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e19301f6dc8c4ed1815a8ab00125c8e5', 'request_id': 'req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:47.387491 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] The token cache is being invalidate because project e19301f6dc8c4ed1815a8ab00125c8e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:40:47.387775 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:47.388385 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:47.389880 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1435/2876] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:40:47 2022] DELETE /identity/v3/projects/e19301f6dc8c4ed1815a8ab00125c8e5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:56.940045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9e2a7-6598-419b-a093-b5d53c5e4cc0 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:56.941802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-93e9e2a7-6598-419b-a093-b5d53c5e4cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:56.955069 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9e2a7-6598-419b-a093-b5d53c5e4cc0 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:57.018627 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9aedc-64de-4842-9cf0-d9695bc60b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:57.019272 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9aedc-64de-4842-9cf0-d9695bc60b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:57.019546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9aedc-64de-4842-9cf0-d9695bc60b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:57.019859 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9aedc-64de-4842-9cf0-d9695bc60b3b admin admin] PATH_INFO: `/v3/users/4a4b858db3ae40fe9a64bc447d2f589c` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:57.021106 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9aedc-64de-4842-9cf0-d9695bc60b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a4b858db3ae40fe9a64bc447d2f589c)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:57.023164 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9aedc-64de-4842-9cf0-d9695bc60b3b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:57.028398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:57.073118 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bff9aedc-64de-4842-9cf0-d9695bc60b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a4b858db3ae40fe9a64bc447d2f589c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.074090 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bff9aedc-64de-4842-9cf0-d9695bc60b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a4b858db3ae40fe9a64bc447d2f589c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.074681 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bff9aedc-64de-4842-9cf0-d9695bc60b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a4b858db3ae40fe9a64bc447d2f589c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.081792 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bff9aedc-64de-4842-9cf0-d9695bc60b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a4b858db3ae40fe9a64bc447d2f589c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.095884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-bff9aedc-64de-4842-9cf0-d9695bc60b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a4b858db3ae40fe9a64bc447d2f589c'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.110652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1442/2877] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:40:56 2022] DELETE /identity/v3/users/4a4b858db3ae40fe9a64bc447d2f589c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:57.286459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:57.288187 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:57.317782 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05ba12-92b3-4ba0-8e4c-5248ee212ca9 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:57.381324 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:57.381995 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:57.382267 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:57.382560 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] PATH_INFO: `/v3/projects/fd68e82b15a148b382fed0455e767628` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:57.383715 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd68e82b15a148b382fed0455e767628)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:57.385629 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:57.443078 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd68e82b15a148b382fed0455e767628', 'request_id': 'req-d010c8d4-5611-4ce5-99af-296f6b14b607'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.449364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd68e82b15a148b382fed0455e767628', 'request_id': 'req-d010c8d4-5611-4ce5-99af-296f6b14b607'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.461020 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd68e82b15a148b382fed0455e767628', 'request_id': 'req-d010c8d4-5611-4ce5-99af-296f6b14b607'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.471532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] The token cache is being invalidate because project fd68e82b15a148b382fed0455e767628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:40:57.471812 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.472453 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.474237 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1436/2878] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:40:57 2022] DELETE /identity/v3/projects/fd68e82b15a148b382fed0455e767628 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:57.478180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9aedc-64de-4842-9cf0-d9695bc60b3b admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:57.479652 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-bff9aedc-64de-4842-9cf0-d9695bc60b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:57.492821 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9aedc-64de-4842-9cf0-d9695bc60b3b admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:57.551546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1db31-38db-4784-b283-fb09bd86e87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:57.552168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1db31-38db-4784-b283-fb09bd86e87b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:57.552443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1db31-38db-4784-b283-fb09bd86e87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:57.552695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1db31-38db-4784-b283-fb09bd86e87b admin admin] PATH_INFO: `/v3/users/331bd879dc7949978d466301fe3f44f4` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:57.553956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1db31-38db-4784-b283-fb09bd86e87b admin admin] RBAC: Authorizing `identity:delete_user(user_id=331bd879dc7949978d466301fe3f44f4)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:57.555956 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1db31-38db-4784-b283-fb09bd86e87b admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:57.607245 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d5e1db31-38db-4784-b283-fb09bd86e87b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '331bd879dc7949978d466301fe3f44f4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.607900 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d5e1db31-38db-4784-b283-fb09bd86e87b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '331bd879dc7949978d466301fe3f44f4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.608505 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d5e1db31-38db-4784-b283-fb09bd86e87b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '331bd879dc7949978d466301fe3f44f4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.615429 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d5e1db31-38db-4784-b283-fb09bd86e87b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '331bd879dc7949978d466301fe3f44f4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.627946 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-d5e1db31-38db-4784-b283-fb09bd86e87b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '331bd879dc7949978d466301fe3f44f4'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.643352 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1443/2879] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:40:57 2022] DELETE /identity/v3/users/331bd879dc7949978d466301fe3f44f4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:57.826272 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:57.827890 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:57.843739 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010c8d4-5611-4ce5-99af-296f6b14b607 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:57.884011 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:57.905130 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293c890-6638-4b0a-9e85-86ece1379f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:57.905699 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293c890-6638-4b0a-9e85-86ece1379f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:57.905973 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293c890-6638-4b0a-9e85-86ece1379f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:57.906238 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293c890-6638-4b0a-9e85-86ece1379f61 admin admin] PATH_INFO: `/v3/projects/3877ced20e18404d81a4958e966d8bfb` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:57.907518 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293c890-6638-4b0a-9e85-86ece1379f61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3877ced20e18404d81a4958e966d8bfb)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:57.909800 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293c890-6638-4b0a-9e85-86ece1379f61 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:57.962239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4293c890-6638-4b0a-9e85-86ece1379f61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3877ced20e18404d81a4958e966d8bfb', 'request_id': 'req-4293c890-6638-4b0a-9e85-86ece1379f61'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.968444 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4293c890-6638-4b0a-9e85-86ece1379f61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3877ced20e18404d81a4958e966d8bfb', 'request_id': 'req-4293c890-6638-4b0a-9e85-86ece1379f61'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.977126 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4293c890-6638-4b0a-9e85-86ece1379f61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3877ced20e18404d81a4958e966d8bfb', 'request_id': 'req-4293c890-6638-4b0a-9e85-86ece1379f61'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.988484 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4293c890-6638-4b0a-9e85-86ece1379f61 admin admin] The token cache is being invalidate because project 3877ced20e18404d81a4958e966d8bfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:40:57.988768 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4293c890-6638-4b0a-9e85-86ece1379f61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.989389 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-4293c890-6638-4b0a-9e85-86ece1379f61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:40:57.991003 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1437/2880] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:40:57 2022] DELETE /identity/v3/projects/3877ced20e18404d81a4958e966d8bfb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:57.997233 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4779e0-32b1-4756-9374-7799d991f22e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:57.997745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4779e0-32b1-4756-9374-7799d991f22e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:57.997745 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4779e0-32b1-4756-9374-7799d991f22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:58.006211 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:58.023596 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-5e4779e0-32b1-4756-9374-7799d991f22e None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:58.025673 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-5e4779e0-32b1-4756-9374-7799d991f22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:58.086664 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1444/2881] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:58.094057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd3150-2e2a-4a73-98b6-12d703899d89 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:58.094338 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd3150-2e2a-4a73-98b6-12d703899d89 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:58.094642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd3150-2e2a-4a73-98b6-12d703899d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:58.115503 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-86cd3150-2e2a-4a73-98b6-12d703899d89 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:58.116712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-86cd3150-2e2a-4a73-98b6-12d703899d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:58.125712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1438/2882] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:58.130066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4779e0-32b1-4756-9374-7799d991f22e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:58.140050 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4779e0-32b1-4756-9374-7799d991f22e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:58.145781 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58e23a-9212-4118-90a9-5ed29b47c0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:58.146409 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58e23a-9212-4118-90a9-5ed29b47c0fe admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:58.146708 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58e23a-9212-4118-90a9-5ed29b47c0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:58.147005 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58e23a-9212-4118-90a9-5ed29b47c0fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:58.147651 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb58e23a-9212-4118-90a9-5ed29b47c0fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:40:58.147950 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb58e23a-9212-4118-90a9-5ed29b47c0fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:58.149584 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb58e23a-9212-4118-90a9-5ed29b47c0fe admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:58.162056 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1445/2883] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:40:58.166300 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd3150-2e2a-4a73-98b6-12d703899d89 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:58.175769 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd3150-2e2a-4a73-98b6-12d703899d89 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:58.181658 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25979b7d-55fc-4207-a1c0-6cad26645635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:58.182246 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25979b7d-55fc-4207-a1c0-6cad26645635 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:58.182529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25979b7d-55fc-4207-a1c0-6cad26645635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:58.182812 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25979b7d-55fc-4207-a1c0-6cad26645635 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:58.183436 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25979b7d-55fc-4207-a1c0-6cad26645635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:58.185313 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25979b7d-55fc-4207-a1c0-6cad26645635 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:58.215991 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1439/2884] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:40:58 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:40:58.220870 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58e23a-9212-4118-90a9-5ed29b47c0fe admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:58.232271 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58e23a-9212-4118-90a9-5ed29b47c0fe admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:58.291391 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85375824-a2db-4bd2-9d49-7db734169bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:58.291995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85375824-a2db-4bd2-9d49-7db734169bdc admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:58.292275 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85375824-a2db-4bd2-9d49-7db734169bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:58.292546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85375824-a2db-4bd2-9d49-7db734169bdc admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:58.293174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85375824-a2db-4bd2-9d49-7db734169bdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:58.295163 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85375824-a2db-4bd2-9d49-7db734169bdc admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:58.318398 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1446/2885] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:40:58 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:40:58.322641 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25979b7d-55fc-4207-a1c0-6cad26645635 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:58.333442 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25979b7d-55fc-4207-a1c0-6cad26645635 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:58.338375 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f56922-ac91-4c93-81c5-b24f574d2f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:58.338998 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f56922-ac91-4c93-81c5-b24f574d2f7f admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:58.339279 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f56922-ac91-4c93-81c5-b24f574d2f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:58.339544 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f56922-ac91-4c93-81c5-b24f574d2f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:58.340188 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f56922-ac91-4c93-81c5-b24f574d2f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:58.341773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f56922-ac91-4c93-81c5-b24f574d2f7f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:58.351983 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1440/2886] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:58.356823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85375824-a2db-4bd2-9d49-7db734169bdc admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:58.372006 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85375824-a2db-4bd2-9d49-7db734169bdc admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:58.377258 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5b57c-07f2-4012-928a-e90c01a0c624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:58.377870 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5b57c-07f2-4012-928a-e90c01a0c624 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:58.378143 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5b57c-07f2-4012-928a-e90c01a0c624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:58.378440 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5b57c-07f2-4012-928a-e90c01a0c624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:58.379237 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5b57c-07f2-4012-928a-e90c01a0c624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:58.380837 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5b57c-07f2-4012-928a-e90c01a0c624 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:58.392672 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1447/2887] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:40:58.397045 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f56922-ac91-4c93-81c5-b24f574d2f7f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:58.407283 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f56922-ac91-4c93-81c5-b24f574d2f7f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:58.413165 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a8ff8-beb5-458f-8dbd-196869086e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:58.413761 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a8ff8-beb5-458f-8dbd-196869086e37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:58.414036 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a8ff8-beb5-458f-8dbd-196869086e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:58.414304 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a8ff8-beb5-458f-8dbd-196869086e37 admin admin] PATH_INFO: `/v3/projects/e2fe3034b0fa4e1698494c4d98b08540/users/b934f2adfe6445e981726c2131f7eb31/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:58.437377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a8ff8-beb5-458f-8dbd-196869086e37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2fe3034b0fa4e1698494c4d98b08540, user_id=b934f2adfe6445e981726c2131f7eb31, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:58.441672 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a8ff8-beb5-458f-8dbd-196869086e37 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:58.455773 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1441/2888] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:40:58 2022] PUT /identity/v3/projects/e2fe3034b0fa4e1698494c4d98b08540/users/b934f2adfe6445e981726c2131f7eb31/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:40:58.466176 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5507a6-5148-46a2-ac7f-3b7b9517b0d7 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:58.466454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5507a6-5148-46a2-ac7f-3b7b9517b0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:58.466751 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5507a6-5148-46a2-ac7f-3b7b9517b0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:58.495527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-6d5507a6-5148-46a2-ac7f-3b7b9517b0d7 None None] MFA Rules not processed for user `b934f2adfe6445e981726c2131f7eb31`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:40:58.496760 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6d5507a6-5148-46a2-ac7f-3b7b9517b0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:58.561035 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:40:58.567330 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1448/2889] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:40:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:40:58.578413 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a8ff8-beb5-458f-8dbd-196869086e37 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:40:58.580302 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-3f9a8ff8-beb5-458f-8dbd-196869086e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:40:58.639735 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a8ff8-beb5-458f-8dbd-196869086e37 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:40:58.643918 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be96ec4-7de7-420a-9910-b0b5d3ca3ea5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:40:58.644513 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96ec4-7de7-420a-9910-b0b5d3ca3ea5 service cinder] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:40:58.644866 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96ec4-7de7-420a-9910-b0b5d3ca3ea5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:40:58.645139 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96ec4-7de7-420a-9910-b0b5d3ca3ea5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:40:58.654042 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be96ec4-7de7-420a-9910-b0b5d3ca3ea5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:40:58.655482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be96ec4-7de7-420a-9910-b0b5d3ca3ea5 service cinder] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:40:58.664683 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1442/2890] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:40:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:41:08.572423 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:41:08.660746 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:41:15.118296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5507a6-5148-46a2-ac7f-3b7b9517b0d7 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:15.191454 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5507a6-5148-46a2-ac7f-3b7b9517b0d7 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:15.196612 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecb8fe-3e46-4209-82e8-5445e8ca4420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:15.197221 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecb8fe-3e46-4209-82e8-5445e8ca4420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:15.197491 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecb8fe-3e46-4209-82e8-5445e8ca4420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:15.197797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecb8fe-3e46-4209-82e8-5445e8ca4420 admin admin] PATH_INFO: `/v3/users/b934f2adfe6445e981726c2131f7eb31` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:15.199010 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecb8fe-3e46-4209-82e8-5445e8ca4420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b934f2adfe6445e981726c2131f7eb31)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:15.200958 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecb8fe-3e46-4209-82e8-5445e8ca4420 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:15.252573 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f0ecb8fe-3e46-4209-82e8-5445e8ca4420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b934f2adfe6445e981726c2131f7eb31'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:15.253343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f0ecb8fe-3e46-4209-82e8-5445e8ca4420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b934f2adfe6445e981726c2131f7eb31'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:15.254066 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f0ecb8fe-3e46-4209-82e8-5445e8ca4420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b934f2adfe6445e981726c2131f7eb31'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:15.260937 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f0ecb8fe-3e46-4209-82e8-5445e8ca4420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b934f2adfe6445e981726c2131f7eb31'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:15.273525 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-f0ecb8fe-3e46-4209-82e8-5445e8ca4420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b934f2adfe6445e981726c2131f7eb31'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:15.292494 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1449/2891] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:41:15 2022] DELETE /identity/v3/users/b934f2adfe6445e981726c2131f7eb31 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:41:15.297382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd679196-c911-4f21-91da-b6783a8396ff None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:15.297885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd679196-c911-4f21-91da-b6783a8396ff None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:15.297885 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd679196-c911-4f21-91da-b6783a8396ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:15.317339 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-dd679196-c911-4f21-91da-b6783a8396ff None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:41:15.318538 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-dd679196-c911-4f21-91da-b6783a8396ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:15.380767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1443/2892] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:41:15.393255 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecb8fe-3e46-4209-82e8-5445e8ca4420 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:15.394812 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f0ecb8fe-3e46-4209-82e8-5445e8ca4420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:15.461308 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecb8fe-3e46-4209-82e8-5445e8ca4420 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:15.465574 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e8e1b-64e3-4dc4-b2a5-aae44062d4b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:15.466174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e8e1b-64e3-4dc4-b2a5-aae44062d4b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:15.466452 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e8e1b-64e3-4dc4-b2a5-aae44062d4b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:15.466753 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e8e1b-64e3-4dc4-b2a5-aae44062d4b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:15.476130 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e8e1b-64e3-4dc4-b2a5-aae44062d4b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:15.477566 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e8e1b-64e3-4dc4-b2a5-aae44062d4b1 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:15.487232 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1450/2893] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:41:15.764395 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd679196-c911-4f21-91da-b6783a8396ff None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:15.765926 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-dd679196-c911-4f21-91da-b6783a8396ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:15.784167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd679196-c911-4f21-91da-b6783a8396ff None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:15.796228 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:15.796881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:15.797156 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:15.797421 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] PATH_INFO: `/v3/projects/e2fe3034b0fa4e1698494c4d98b08540` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:15.798692 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2fe3034b0fa4e1698494c4d98b08540)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:15.800613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:15.850104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2fe3034b0fa4e1698494c4d98b08540', 'request_id': 'req-56cc178c-e47a-45e0-a132-51763a04722f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:15.856606 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2fe3034b0fa4e1698494c4d98b08540', 'request_id': 'req-56cc178c-e47a-45e0-a132-51763a04722f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:15.866654 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2fe3034b0fa4e1698494c4d98b08540', 'request_id': 'req-56cc178c-e47a-45e0-a132-51763a04722f'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:15.877241 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] The token cache is being invalidate because project e2fe3034b0fa4e1698494c4d98b08540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:41:15.877519 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:15.878132 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:15.879767 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1444/2894] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:41:15 2022] DELETE /identity/v3/projects/e2fe3034b0fa4e1698494c4d98b08540 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:41:23.746869 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453af86-9d7e-4dff-b81d-77eb633b77aa None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:23.747328 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453af86-9d7e-4dff-b81d-77eb633b77aa None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:23.747425 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453af86-9d7e-4dff-b81d-77eb633b77aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:23.766914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c453af86-9d7e-4dff-b81d-77eb633b77aa None None] MFA Rules not processed for user `048b4abace244e48b6897daec0277e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:41:23.768104 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c453af86-9d7e-4dff-b81d-77eb633b77aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:23.785798 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:41:23.837026 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1451/2895] 10.222.0.23 () {58 vars in 967 bytes} [Fri Sep 9 16:41:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:41:23.882875 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:23.884520 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:23.957158 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc178c-e47a-45e0-a132-51763a04722f admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:23.962689 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8dabe-d6c0-4aed-8891-cf0a8724d414 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e926034945b44087b378b3468954a18a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:23.963218 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8dabe-d6c0-4aed-8891-cf0a8724d414 service swift] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:23.963490 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8dabe-d6c0-4aed-8891-cf0a8724d414 service swift] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:23.963756 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8dabe-d6c0-4aed-8891-cf0a8724d414 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:23.972749 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8dabe-d6c0-4aed-8891-cf0a8724d414 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:23.974330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8dabe-d6c0-4aed-8891-cf0a8724d414 service swift] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:23.982586 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1445/2896] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Sep 9 16:41:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 16:41:30.656065 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453af86-9d7e-4dff-b81d-77eb633b77aa None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:30.657636 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c453af86-9d7e-4dff-b81d-77eb633b77aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:30.725938 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453af86-9d7e-4dff-b81d-77eb633b77aa None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:30.731092 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631fe3a1-2717-4b66-b418-24016fb5dfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:30.731786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fe3a1-2717-4b66-b418-24016fb5dfa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:30.732063 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fe3a1-2717-4b66-b418-24016fb5dfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:30.732336 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fe3a1-2717-4b66-b418-24016fb5dfa6 admin admin] PATH_INFO: `/v3/users/f86d1ec821814f628879590513702cec` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:30.733463 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631fe3a1-2717-4b66-b418-24016fb5dfa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f86d1ec821814f628879590513702cec)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:30.735457 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631fe3a1-2717-4b66-b418-24016fb5dfa6 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:30.791167 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-631fe3a1-2717-4b66-b418-24016fb5dfa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f86d1ec821814f628879590513702cec'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:30.791802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-631fe3a1-2717-4b66-b418-24016fb5dfa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f86d1ec821814f628879590513702cec'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:30.792469 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-631fe3a1-2717-4b66-b418-24016fb5dfa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f86d1ec821814f628879590513702cec'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:30.801953 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-631fe3a1-2717-4b66-b418-24016fb5dfa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f86d1ec821814f628879590513702cec'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:30.815620 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-631fe3a1-2717-4b66-b418-24016fb5dfa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f86d1ec821814f628879590513702cec'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:30.829695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1452/2897] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:41:30 2022] DELETE /identity/v3/users/f86d1ec821814f628879590513702cec => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:41:31.015920 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8dabe-d6c0-4aed-8891-cf0a8724d414 service swift] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:31.027164 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8dabe-d6c0-4aed-8891-cf0a8724d414 service swift] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:31.037384 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:41:31.085620 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:31.086253 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:31.086549 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:31.086818 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] PATH_INFO: `/v3/projects/ff5daf9e9245447dac44278ce813b4a0` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:31.087947 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff5daf9e9245447dac44278ce813b4a0)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:31.089845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:31.144364 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff5daf9e9245447dac44278ce813b4a0', 'request_id': 'req-360d30c5-8487-442c-ba9b-4008b7a98e59'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.151470 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff5daf9e9245447dac44278ce813b4a0', 'request_id': 'req-360d30c5-8487-442c-ba9b-4008b7a98e59'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.160738 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff5daf9e9245447dac44278ce813b4a0', 'request_id': 'req-360d30c5-8487-442c-ba9b-4008b7a98e59'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.172372 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] The token cache is being invalidate because project ff5daf9e9245447dac44278ce813b4a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:41:31.172712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.173331 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.174881 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1446/2898] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:41:31 2022] DELETE /identity/v3/projects/ff5daf9e9245447dac44278ce813b4a0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:41:31.179034 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631fe3a1-2717-4b66-b418-24016fb5dfa6 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:31.180501 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-631fe3a1-2717-4b66-b418-24016fb5dfa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:31.193584 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631fe3a1-2717-4b66-b418-24016fb5dfa6 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:31.232364 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:41:31.253508 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7f2ed-e4e3-4a66-925d-38f9636a4614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:31.254137 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7f2ed-e4e3-4a66-925d-38f9636a4614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:31.254414 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7f2ed-e4e3-4a66-925d-38f9636a4614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:31.254736 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7f2ed-e4e3-4a66-925d-38f9636a4614 admin admin] PATH_INFO: `/v3/users/5eb42a654af74ea59f0bd614db3c3c0d` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:31.255934 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7f2ed-e4e3-4a66-925d-38f9636a4614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eb42a654af74ea59f0bd614db3c3c0d)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:31.257930 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7f2ed-e4e3-4a66-925d-38f9636a4614 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:31.308749 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a9b7f2ed-e4e3-4a66-925d-38f9636a4614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eb42a654af74ea59f0bd614db3c3c0d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.309478 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a9b7f2ed-e4e3-4a66-925d-38f9636a4614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eb42a654af74ea59f0bd614db3c3c0d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.310123 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a9b7f2ed-e4e3-4a66-925d-38f9636a4614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eb42a654af74ea59f0bd614db3c3c0d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.317460 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a9b7f2ed-e4e3-4a66-925d-38f9636a4614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eb42a654af74ea59f0bd614db3c3c0d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.329414 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-a9b7f2ed-e4e3-4a66-925d-38f9636a4614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eb42a654af74ea59f0bd614db3c3c0d'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.343852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1453/2899] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:41:31 2022] DELETE /identity/v3/users/5eb42a654af74ea59f0bd614db3c3c0d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:41:31.514948 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:31.516561 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:31.529549 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d30c5-8487-442c-ba9b-4008b7a98e59 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:31.585921 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85590635-d7f3-405b-8a70-4fc1c6eb233e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:31.586598 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85590635-d7f3-405b-8a70-4fc1c6eb233e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:31.586878 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85590635-d7f3-405b-8a70-4fc1c6eb233e admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:31.587151 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85590635-d7f3-405b-8a70-4fc1c6eb233e admin admin] PATH_INFO: `/v3/projects/a95b4d8c1df849c68e45fe23c0eca31a` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:31.588257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85590635-d7f3-405b-8a70-4fc1c6eb233e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a95b4d8c1df849c68e45fe23c0eca31a)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:31.590210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85590635-d7f3-405b-8a70-4fc1c6eb233e admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:31.638427 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:41:31.641557 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-85590635-d7f3-405b-8a70-4fc1c6eb233e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a95b4d8c1df849c68e45fe23c0eca31a', 'request_id': 'req-85590635-d7f3-405b-8a70-4fc1c6eb233e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.648964 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-85590635-d7f3-405b-8a70-4fc1c6eb233e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a95b4d8c1df849c68e45fe23c0eca31a', 'request_id': 'req-85590635-d7f3-405b-8a70-4fc1c6eb233e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.657152 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-85590635-d7f3-405b-8a70-4fc1c6eb233e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a95b4d8c1df849c68e45fe23c0eca31a', 'request_id': 'req-85590635-d7f3-405b-8a70-4fc1c6eb233e'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.668122 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-85590635-d7f3-405b-8a70-4fc1c6eb233e admin admin] The token cache is being invalidate because project a95b4d8c1df849c68e45fe23c0eca31a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81510) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:41:31.668420 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-85590635-d7f3-405b-8a70-4fc1c6eb233e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.669112 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-85590635-d7f3-405b-8a70-4fc1c6eb233e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:41:31.670644 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1447/2900] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:41:31 2022] DELETE /identity/v3/projects/a95b4d8c1df849c68e45fe23c0eca31a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:41:31.675914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d1382-0bb3-4cf5-bccd-432cddd481ab None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:31.676405 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d1382-0bb3-4cf5-bccd-432cddd481ab None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:31.676519 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d1382-0bb3-4cf5-bccd-432cddd481ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:31.697162 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-c46d1382-0bb3-4cf5-bccd-432cddd481ab None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:41:31.698723 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c46d1382-0bb3-4cf5-bccd-432cddd481ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:31.761180 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1454/2901] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:41:31.769588 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55564986-19ce-4c54-8ea8-33435c2e2ba4 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:31.769896 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55564986-19ce-4c54-8ea8-33435c2e2ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:31.770167 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55564986-19ce-4c54-8ea8-33435c2e2ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:31.791845 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-55564986-19ce-4c54-8ea8-33435c2e2ba4 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:41:31.793061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-55564986-19ce-4c54-8ea8-33435c2e2ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:31.803143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1448/2902] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:41:31.807490 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d1382-0bb3-4cf5-bccd-432cddd481ab None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:31.817923 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d1382-0bb3-4cf5-bccd-432cddd481ab None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:31.824470 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6689d5-da85-423e-989f-540877a87cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:31.825147 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6689d5-da85-423e-989f-540877a87cef admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:31.825436 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6689d5-da85-423e-989f-540877a87cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:31.825680 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6689d5-da85-423e-989f-540877a87cef admin admin] PATH_INFO: `/v3/domains` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:31.826332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6689d5-da85-423e-989f-540877a87cef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81511) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 16:41:31.826670 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6689d5-da85-423e-989f-540877a87cef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:31.828321 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6689d5-da85-423e-989f-540877a87cef admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:31.840222 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1455/2903] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Sep 9 16:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 16:41:31.844542 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55564986-19ce-4c54-8ea8-33435c2e2ba4 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:31.854019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55564986-19ce-4c54-8ea8-33435c2e2ba4 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:31.859788 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d53f7e-744b-4fa3-824a-03a71f354afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:31.860397 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53f7e-744b-4fa3-824a-03a71f354afb admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:31.860673 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53f7e-744b-4fa3-824a-03a71f354afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:31.860939 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53f7e-744b-4fa3-824a-03a71f354afb admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:31.861589 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d53f7e-744b-4fa3-824a-03a71f354afb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:31.863642 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d53f7e-744b-4fa3-824a-03a71f354afb admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:31.901143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1449/2904] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:41:31 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:41:31.905832 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6689d5-da85-423e-989f-540877a87cef admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:31.918699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6689d5-da85-423e-989f-540877a87cef admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:31.967289 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:41:31.983868 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2885c92-fa9f-45d5-afa2-2b2149aba3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:31.984478 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2885c92-fa9f-45d5-afa2-2b2149aba3bd admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:31.984751 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2885c92-fa9f-45d5-afa2-2b2149aba3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:31.985018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2885c92-fa9f-45d5-afa2-2b2149aba3bd admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:31.985653 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2885c92-fa9f-45d5-afa2-2b2149aba3bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:31.987659 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2885c92-fa9f-45d5-afa2-2b2149aba3bd admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:32.009947 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1456/2905] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:41:31 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:41:32.014630 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d53f7e-744b-4fa3-824a-03a71f354afb admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:32.025509 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d53f7e-744b-4fa3-824a-03a71f354afb admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:32.030652 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c0eaf-b065-4d40-8372-713ee2eda924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:32.031257 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c0eaf-b065-4d40-8372-713ee2eda924 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:32.031532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c0eaf-b065-4d40-8372-713ee2eda924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:32.031798 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c0eaf-b065-4d40-8372-713ee2eda924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:32.032439 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c0eaf-b065-4d40-8372-713ee2eda924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:32.034382 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c0eaf-b065-4d40-8372-713ee2eda924 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:32.044463 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1450/2906] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:41:32.048712 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2885c92-fa9f-45d5-afa2-2b2149aba3bd admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:32.059174 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2885c92-fa9f-45d5-afa2-2b2149aba3bd admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:32.064273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb864b-f420-41ef-a24a-98a0f4a9ec82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:32.064880 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb864b-f420-41ef-a24a-98a0f4a9ec82 admin admin] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:32.065149 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb864b-f420-41ef-a24a-98a0f4a9ec82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:32.065419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb864b-f420-41ef-a24a-98a0f4a9ec82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:32.066089 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb864b-f420-41ef-a24a-98a0f4a9ec82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:32.067734 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb864b-f420-41ef-a24a-98a0f4a9ec82 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:32.078635 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1457/2907] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:41:32.082867 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c0eaf-b065-4d40-8372-713ee2eda924 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:32.093523 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c0eaf-b065-4d40-8372-713ee2eda924 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:32.099001 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c20bcc-4063-4877-ba32-0af44e541cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:32.099682 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c20bcc-4063-4877-ba32-0af44e541cda admin admin] REQUEST_METHOD: `PUT` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:32.099961 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c20bcc-4063-4877-ba32-0af44e541cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:32.100233 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c20bcc-4063-4877-ba32-0af44e541cda admin admin] PATH_INFO: `/v3/projects/5c52563210cf435696f3e3ae3907b0fb/users/69ec3d4f006c4f8f815b0cad9f933208/roles/9e02fa627e5c4388b0aa7182f1707895` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:32.115626 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c20bcc-4063-4877-ba32-0af44e541cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c52563210cf435696f3e3ae3907b0fb, user_id=69ec3d4f006c4f8f815b0cad9f933208, role_id=9e02fa627e5c4388b0aa7182f1707895)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:32.118285 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c20bcc-4063-4877-ba32-0af44e541cda admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:32.132430 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1451/2908] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:41:32 2022] PUT /identity/v3/projects/5c52563210cf435696f3e3ae3907b0fb/users/69ec3d4f006c4f8f815b0cad9f933208/roles/9e02fa627e5c4388b0aa7182f1707895 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 16:41:32.138449 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b9eb9-c7f5-4942-a6f6-3b7adf546ba8 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:32.138852 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b9eb9-c7f5-4942-a6f6-3b7adf546ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:32.139242 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b9eb9-c7f5-4942-a6f6-3b7adf546ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:32.162639 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-7c6b9eb9-c7f5-4942-a6f6-3b7adf546ba8 None None] MFA Rules not processed for user `94cbbd1503324af1af6df581aea570b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:41:32.163842 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7c6b9eb9-c7f5-4942-a6f6-3b7adf546ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:32.228583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1458/2909] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 16:41:32.238471 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c20bcc-4063-4877-ba32-0af44e541cda admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:32.240381 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-e7c20bcc-4063-4877-ba32-0af44e541cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:32.298124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c20bcc-4063-4877-ba32-0af44e541cda admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:32.302205 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebea8fe-bb46-44e1-83b5-ccd83504fe78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:32.302839 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebea8fe-bb46-44e1-83b5-ccd83504fe78 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:32.303108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebea8fe-bb46-44e1-83b5-ccd83504fe78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:32.303377 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebea8fe-bb46-44e1-83b5-ccd83504fe78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:32.312659 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebea8fe-bb46-44e1-83b5-ccd83504fe78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:32.314076 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebea8fe-bb46-44e1-83b5-ccd83504fe78 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:32.323435 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1452/2910] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:41:36.250824 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2863116-3572-4621-b4c0-33a773ca467e None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:36.251499 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2863116-3572-4621-b4c0-33a773ca467e None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:36.251499 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2863116-3572-4621-b4c0-33a773ca467e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:36.281797 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-b2863116-3572-4621-b4c0-33a773ca467e None None] MFA Rules not processed for user `69ec3d4f006c4f8f815b0cad9f933208`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:41:36.283546 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-b2863116-3572-4621-b4c0-33a773ca467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:36.353303 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1459/2911] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:41:36.358185 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebea8fe-bb46-44e1-83b5-ccd83504fe78 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:36.368320 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebea8fe-bb46-44e1-83b5-ccd83504fe78 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:36.373310 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43feef-69cb-482c-b56f-40d526f58867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:36.374142 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43feef-69cb-482c-b56f-40d526f58867 admin admin] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:36.374415 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43feef-69cb-482c-b56f-40d526f58867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:36.374716 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43feef-69cb-482c-b56f-40d526f58867 admin admin] PATH_INFO: `/v3/projects` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:36.375351 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43feef-69cb-482c-b56f-40d526f58867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:36.377298 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43feef-69cb-482c-b56f-40d526f58867 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:36.419191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1453/2912] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Sep 9 16:41:36 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:41:36.423461 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2863116-3572-4621-b4c0-33a773ca467e None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:36.434823 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2863116-3572-4621-b4c0-33a773ca467e None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:36.485701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:41:36.493785 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59332f-ad0c-4081-9723-881a0ec7ef26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:36.494402 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59332f-ad0c-4081-9723-881a0ec7ef26 admin admin] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:36.494750 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59332f-ad0c-4081-9723-881a0ec7ef26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:36.495020 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59332f-ad0c-4081-9723-881a0ec7ef26 admin admin] PATH_INFO: `/v3/users` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:36.495661 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59332f-ad0c-4081-9723-881a0ec7ef26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:36.497650 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59332f-ad0c-4081-9723-881a0ec7ef26 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:36.522997 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1460/2913] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Sep 9 16:41:36 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 16:41:36.527081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43feef-69cb-482c-b56f-40d526f58867 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:36.538459 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43feef-69cb-482c-b56f-40d526f58867 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:36.543259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97096b-5e88-4164-bb62-dadf3b2d2e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:36.543940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97096b-5e88-4164-bb62-dadf3b2d2e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:36.544213 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97096b-5e88-4164-bb62-dadf3b2d2e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:36.544480 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97096b-5e88-4164-bb62-dadf3b2d2e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:36.545127 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97096b-5e88-4164-bb62-dadf3b2d2e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:36.546764 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97096b-5e88-4164-bb62-dadf3b2d2e82 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:36.557466 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1454/2914] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:41:36.561841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59332f-ad0c-4081-9723-881a0ec7ef26 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:36.572914 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59332f-ad0c-4081-9723-881a0ec7ef26 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:36.578228 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6043a2e-3e4b-4320-b026-d0ceb3d53d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:36.578980 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6043a2e-3e4b-4320-b026-d0ceb3d53d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:36.579259 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6043a2e-3e4b-4320-b026-d0ceb3d53d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:36.579533 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6043a2e-3e4b-4320-b026-d0ceb3d53d51 admin admin] PATH_INFO: `/v3/projects/902c3f9c73124ba3af7787d3a20ed898/users/5b384779760949de996835ac355e5179/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:36.595686 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6043a2e-3e4b-4320-b026-d0ceb3d53d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=902c3f9c73124ba3af7787d3a20ed898, user_id=5b384779760949de996835ac355e5179, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:36.598484 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6043a2e-3e4b-4320-b026-d0ceb3d53d51 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:36.614222 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1461/2915] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:41:36 2022] PUT /identity/v3/projects/902c3f9c73124ba3af7787d3a20ed898/users/5b384779760949de996835ac355e5179/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:41:36.618236 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97096b-5e88-4164-bb62-dadf3b2d2e82 admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:36.628543 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97096b-5e88-4164-bb62-dadf3b2d2e82 admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:36.658124 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:41:36.694868 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3910cc-64d3-4fa5-aabc-c8885b70a4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:36.695493 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3910cc-64d3-4fa5-aabc-c8885b70a4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:36.695771 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3910cc-64d3-4fa5-aabc-c8885b70a4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:36.696041 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3910cc-64d3-4fa5-aabc-c8885b70a4b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:36.696711 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3910cc-64d3-4fa5-aabc-c8885b70a4b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:36.698322 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3910cc-64d3-4fa5-aabc-c8885b70a4b4 admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:36.708662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1455/2916] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Sep 9 16:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 16:41:36.712966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6043a2e-3e4b-4320-b026-d0ceb3d53d51 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:36.723919 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6043a2e-3e4b-4320-b026-d0ceb3d53d51 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:36.728843 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25edfe3-26b4-4249-b2e9-36afe10321b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:36.729528 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25edfe3-26b4-4249-b2e9-36afe10321b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:36.729804 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25edfe3-26b4-4249-b2e9-36afe10321b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:36.730077 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25edfe3-26b4-4249-b2e9-36afe10321b9 admin admin] PATH_INFO: `/v3/projects/902c3f9c73124ba3af7787d3a20ed898/users/5b384779760949de996835ac355e5179/roles/c6da7ba2be504943af29338ee8855f05` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:36.732146 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25edfe3-26b4-4249-b2e9-36afe10321b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=902c3f9c73124ba3af7787d3a20ed898, user_id=5b384779760949de996835ac355e5179, role_id=c6da7ba2be504943af29338ee8855f05)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:36.734848 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25edfe3-26b4-4249-b2e9-36afe10321b9 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:36.746096 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1462/2917] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Sep 9 16:41:36 2022] PUT /identity/v3/projects/902c3f9c73124ba3af7787d3a20ed898/users/5b384779760949de996835ac355e5179/roles/c6da7ba2be504943af29338ee8855f05 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:41:41.270228 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a84c9-a13d-463c-894f-8b9fb3f8c6a8 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:41.271096 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a84c9-a13d-463c-894f-8b9fb3f8c6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:41.271537 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a84c9-a13d-463c-894f-8b9fb3f8c6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:41.304637 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-4f5a84c9-a13d-463c-894f-8b9fb3f8c6a8 None None] MFA Rules not processed for user `5b384779760949de996835ac355e5179`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:41:41.307790 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-4f5a84c9-a13d-463c-894f-8b9fb3f8c6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:41.387437 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1456/2918] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:41:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:41:41.403383 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25edfe3-26b4-4249-b2e9-36afe10321b9 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:41.406375 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-c25edfe3-26b4-4249-b2e9-36afe10321b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:41.509311 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25edfe3-26b4-4249-b2e9-36afe10321b9 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:41.515045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbab095-9dde-44d6-a27d-46627582bc24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:41.515664 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbab095-9dde-44d6-a27d-46627582bc24 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:41.515941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbab095-9dde-44d6-a27d-46627582bc24 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:41.516209 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbab095-9dde-44d6-a27d-46627582bc24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:41.575941 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbab095-9dde-44d6-a27d-46627582bc24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:41.577584 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbab095-9dde-44d6-a27d-46627582bc24 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:41.586992 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1463/2919] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:41:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:41:51.382832 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=49 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:41:51.582873 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=84,keystone:INSERT=5 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:41:56.891988 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67c070-d621-46bb-b6b1-79b17e089a34 None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:56.892529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67c070-d621-46bb-b6b1-79b17e089a34 None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:56.892529 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67c070-d621-46bb-b6b1-79b17e089a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:56.912104 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-1b67c070-d621-46bb-b6b1-79b17e089a34 None None] MFA Rules not processed for user `69ec3d4f006c4f8f815b0cad9f933208`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:41:56.913299 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1b67c070-d621-46bb-b6b1-79b17e089a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:56.920816 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1457/2920] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:41:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:41:56.970619 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbab095-9dde-44d6-a27d-46627582bc24 service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:56.972600 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-dfbab095-9dde-44d6-a27d-46627582bc24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:57.033665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbab095-9dde-44d6-a27d-46627582bc24 service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:57.037675 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614d161-e5cd-4d58-8648-f0e3959d6663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b665b9e27144b5bd684ece7ff465e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:57.038273 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614d161-e5cd-4d58-8648-f0e3959d6663 service cinder] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:57.038552 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614d161-e5cd-4d58-8648-f0e3959d6663 service cinder] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:57.038839 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614d161-e5cd-4d58-8648-f0e3959d6663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:57.040841 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-6614d161-e5cd-4d58-8648-f0e3959d6663 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:57.054922 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614d161-e5cd-4d58-8648-f0e3959d6663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:57.056351 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614d161-e5cd-4d58-8648-f0e3959d6663 service cinder] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:57.067239 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1464/2921] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:41:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:41:58.829137 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67c070-d621-46bb-b6b1-79b17e089a34 None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:41:58.830723 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1b67c070-d621-46bb-b6b1-79b17e089a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:58.886799 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67c070-d621-46bb-b6b1-79b17e089a34 None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:41:58.890443 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65830e05-d7b4-49ec-a739-9ca5740aacab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:41:58.891081 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65830e05-d7b4-49ec-a739-9ca5740aacab service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:41:58.891354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65830e05-d7b4-49ec-a739-9ca5740aacab service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:41:58.891629 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65830e05-d7b4-49ec-a739-9ca5740aacab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:41:58.893396 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-65830e05-d7b4-49ec-a739-9ca5740aacab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:41:58.907172 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65830e05-d7b4-49ec-a739-9ca5740aacab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:41:58.908949 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65830e05-d7b4-49ec-a739-9ca5740aacab service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:41:58.920210 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1458/2922] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:41:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 16:42:07.062669 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=26 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:42:08.916061 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=29 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:42:16.168665 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614d161-e5cd-4d58-8648-f0e3959d6663 service cinder] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:42:16.181645 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614d161-e5cd-4d58-8648-f0e3959d6663 service cinder] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:42:16.185872 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fbbf7-217d-402e-b86d-114f038387d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:42:16.186473 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fbbf7-217d-402e-b86d-114f038387d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:42:16.186800 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fbbf7-217d-402e-b86d-114f038387d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:42:16.187063 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fbbf7-217d-402e-b86d-114f038387d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:42:16.197233 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fbbf7-217d-402e-b86d-114f038387d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:42:16.198701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fbbf7-217d-402e-b86d-114f038387d5 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:42:16.208820 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1465/2923] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:42:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:42:26.204620 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:43:00.103742 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65830e05-d7b4-49ec-a739-9ca5740aacab service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:43:00.113808 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65830e05-d7b4-49ec-a739-9ca5740aacab service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:43:00.118432 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba16bc5-e7ab-4f62-b84d-48979279ab0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:43:00.119069 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba16bc5-e7ab-4f62-b84d-48979279ab0d service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:43:00.119348 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba16bc5-e7ab-4f62-b84d-48979279ab0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:43:00.119615 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba16bc5-e7ab-4f62-b84d-48979279ab0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:43:00.121720 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-dba16bc5-e7ab-4f62-b84d-48979279ab0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:43:00.131927 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba16bc5-e7ab-4f62-b84d-48979279ab0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:43:00.133354 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba16bc5-e7ab-4f62-b84d-48979279ab0d service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:43:00.141712 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1459/2924] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:43:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:43:10.138143 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=6 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:44:15.383217 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fbbf7-217d-402e-b86d-114f038387d5 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:44:15.396909 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fbbf7-217d-402e-b86d-114f038387d5 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:44:15.402350 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc085f-5014-400a-9e70-80bbfc505bcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:44:15.402979 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc085f-5014-400a-9e70-80bbfc505bcd service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:44:15.403276 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc085f-5014-400a-9e70-80bbfc505bcd service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:44:15.403549 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc085f-5014-400a-9e70-80bbfc505bcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:44:15.405936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a1fc085f-5014-400a-9e70-80bbfc505bcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:44:15.423678 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc085f-5014-400a-9e70-80bbfc505bcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:44:15.425296 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc085f-5014-400a-9e70-80bbfc505bcd service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:44:15.437809 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1466/2925] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:44:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:44:16.589251 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b21e5-bbaf-4742-9f05-1fbfdefe7aee None None] REQUEST_METHOD: `POST` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:44:16.589826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b21e5-bbaf-4742-9f05-1fbfdefe7aee None None] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:44:16.589826 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b21e5-bbaf-4742-9f05-1fbfdefe7aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:44:16.610923 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.auth.core [None req-6b6b21e5-bbaf-4742-9f05-1fbfdefe7aee None None] MFA Rules not processed for user `69ec3d4f006c4f8f815b0cad9f933208`. Rule list: `[]` (Enabled: `True`). {{(pid=81510) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:44:16.612049 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-6b6b21e5-bbaf-4742-9f05-1fbfdefe7aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:44:16.619774 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1460/2926] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:44:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:44:16.951527 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc085f-5014-400a-9e70-80bbfc505bcd service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:44:16.966404 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc085f-5014-400a-9e70-80bbfc505bcd service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:44:16.972192 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10548e15-5ca6-436d-a63c-db7da2d890c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:44:16.972791 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10548e15-5ca6-436d-a63c-db7da2d890c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:44:16.973072 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10548e15-5ca6-436d-a63c-db7da2d890c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:44:16.973343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10548e15-5ca6-436d-a63c-db7da2d890c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:44:16.975516 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-10548e15-5ca6-436d-a63c-db7da2d890c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:44:16.985897 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10548e15-5ca6-436d-a63c-db7da2d890c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:44:16.987409 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10548e15-5ca6-436d-a63c-db7da2d890c9 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:44:16.995293 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1467/2927] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:44:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:44:17.479385 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b21e5-bbaf-4742-9f05-1fbfdefe7aee None None] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:44:17.481039 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-6b6b21e5-bbaf-4742-9f05-1fbfdefe7aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:44:17.555539 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b21e5-bbaf-4742-9f05-1fbfdefe7aee None None] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:44:17.560651 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e63be-b418-47e0-9aa6-a3bb7cd38031 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784b6147de91435292e4cf3940bcad9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:44:17.561259 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e63be-b418-47e0-9aa6-a3bb7cd38031 service placement] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:44:17.561531 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e63be-b418-47e0-9aa6-a3bb7cd38031 service placement] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:44:17.561793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e63be-b418-47e0-9aa6-a3bb7cd38031 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:44:17.563686 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-651e63be-b418-47e0-9aa6-a3bb7cd38031 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:44:17.574756 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e63be-b418-47e0-9aa6-a3bb7cd38031 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:44:17.576316 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e63be-b418-47e0-9aa6-a3bb7cd38031 service placement] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:44:17.585945 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1461/2928] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:44:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:44:26.991832 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=12 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:44:27.582108 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=35 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:44:30.550047 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10548e15-5ca6-436d-a63c-db7da2d890c9 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:44:30.565510 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10548e15-5ca6-436d-a63c-db7da2d890c9 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:44:30.571692 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442e147-3f57-4be2-901a-5139d47b2fd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:44:30.572247 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442e147-3f57-4be2-901a-5139d47b2fd0 service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:44:30.572419 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442e147-3f57-4be2-901a-5139d47b2fd0 service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:44:30.572695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442e147-3f57-4be2-901a-5139d47b2fd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:44:30.574630 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f442e147-3f57-4be2-901a-5139d47b2fd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:44:30.587605 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442e147-3f57-4be2-901a-5139d47b2fd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:44:30.589267 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442e147-3f57-4be2-901a-5139d47b2fd0 service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:44:30.601244 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1468/2929] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:44:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:44:40.596410 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:45:03.366411 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e63be-b418-47e0-9aa6-a3bb7cd38031 service placement] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:45:03.377583 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e63be-b418-47e0-9aa6-a3bb7cd38031 service placement] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:45:03.384019 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19a683-c746-4953-bf4b-96ffed029c6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784b6147de91435292e4cf3940bcad9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:45:03.384613 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19a683-c746-4953-bf4b-96ffed029c6b service placement] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:45:03.384893 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19a683-c746-4953-bf4b-96ffed029c6b service placement] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:45:03.385154 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19a683-c746-4953-bf4b-96ffed029c6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:45:03.387278 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-2f19a683-c746-4953-bf4b-96ffed029c6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:45:03.399149 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19a683-c746-4953-bf4b-96ffed029c6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:45:03.400722 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19a683-c746-4953-bf4b-96ffed029c6b service placement] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:45:03.413129 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1462/2930] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:45:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:45:13.409057 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=6 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:46:08.458392 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1879a48-c084-4f00-9426-83cdc68361a0 None None] REQUEST_METHOD: `POST` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:46:08.458963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1879a48-c084-4f00-9426-83cdc68361a0 None None] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:46:08.458963 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1879a48-c084-4f00-9426-83cdc68361a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:46:08.482260 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.auth.core [None req-a1879a48-c084-4f00-9426-83cdc68361a0 None None] MFA Rules not processed for user `69ec3d4f006c4f8f815b0cad9f933208`. Rule list: `[]` (Enabled: `True`). {{(pid=81511) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 16:46:08.484213 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-a1879a48-c084-4f00-9426-83cdc68361a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:46:08.494133 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1469/2931] 10.222.0.23 () {58 vars in 958 bytes} [Fri Sep 9 16:46:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 16:46:08.823179 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19a683-c746-4953-bf4b-96ffed029c6b service placement] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:46:08.833239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19a683-c746-4953-bf4b-96ffed029c6b service placement] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:46:08.840240 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607dc97-34e3-4e57-83d9-7239a0d57b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:46:08.840822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607dc97-34e3-4e57-83d9-7239a0d57b79 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:46:08.841118 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607dc97-34e3-4e57-83d9-7239a0d57b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:46:08.841380 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607dc97-34e3-4e57-83d9-7239a0d57b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:46:08.843524 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-0607dc97-34e3-4e57-83d9-7239a0d57b79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:46:08.854235 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607dc97-34e3-4e57-83d9-7239a0d57b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:46:08.855695 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607dc97-34e3-4e57-83d9-7239a0d57b79 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:46:08.864638 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1463/2932] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:46:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:46:18.474966 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=3 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:46:18.860253 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=6 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:46:31.387884 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1879a48-c084-4f00-9426-83cdc68361a0 None None] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:46:31.401332 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1879a48-c084-4f00-9426-83cdc68361a0 None None] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:46:31.407443 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d5d53-a9d1-4ab3-b6fe-6d03b744308e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152a41ad598b4b01b1881d928c53c848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:46:31.408059 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d5d53-a9d1-4ab3-b6fe-6d03b744308e service nova] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:46:31.408335 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d5d53-a9d1-4ab3-b6fe-6d03b744308e service nova] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:46:31.408606 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d5d53-a9d1-4ab3-b6fe-6d03b744308e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:46:31.410412 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-1b2d5d53-a9d1-4ab3-b6fe-6d03b744308e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:46:31.423204 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d5d53-a9d1-4ab3-b6fe-6d03b744308e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:46:31.424763 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d5d53-a9d1-4ab3-b6fe-6d03b744308e service nova] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:46:31.435713 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1470/2933] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:46:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:46:41.431168 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=6 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:48:02.097928 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607dc97-34e3-4e57-83d9-7239a0d57b79 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:48:02.107226 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607dc97-34e3-4e57-83d9-7239a0d57b79 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:48:02.111479 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ee0d9-18f1-4d66-bede-564f1821d8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:48:02.112079 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ee0d9-18f1-4d66-bede-564f1821d8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:48:02.112360 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ee0d9-18f1-4d66-bede-564f1821d8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:48:02.112624 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ee0d9-18f1-4d66-bede-564f1821d8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:48:02.120247 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ee0d9-18f1-4d66-bede-564f1821d8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:48:02.121703 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ee0d9-18f1-4d66-bede-564f1821d8a0 service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:48:02.129770 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1464/2934] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:48:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:48:02.458043 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d5d53-a9d1-4ab3-b6fe-6d03b744308e service nova] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:48:02.468015 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d5d53-a9d1-4ab3-b6fe-6d03b744308e service nova] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:48:02.471789 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600893dc-0408-43f6-ad7a-181b1b207262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:48:02.472407 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600893dc-0408-43f6-ad7a-181b1b207262 service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:48:02.472655 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600893dc-0408-43f6-ad7a-181b1b207262 service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:48:02.472965 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600893dc-0408-43f6-ad7a-181b1b207262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:48:02.482064 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600893dc-0408-43f6-ad7a-181b1b207262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:48:02.483695 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600893dc-0408-43f6-ad7a-181b1b207262 service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:48:02.493098 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1471/2935] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:48:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:48:03.557092 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ee0d9-18f1-4d66-bede-564f1821d8a0 service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:48:03.566294 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ee0d9-18f1-4d66-bede-564f1821d8a0 service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:48:03.569892 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018eda5e-a4be-4bd0-99c2-478ef193e57a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:48:03.570650 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018eda5e-a4be-4bd0-99c2-478ef193e57a service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:48:03.570925 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018eda5e-a4be-4bd0-99c2-478ef193e57a service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:48:03.571239 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018eda5e-a4be-4bd0-99c2-478ef193e57a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:48:03.582191 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018eda5e-a4be-4bd0-99c2-478ef193e57a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:48:03.583696 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018eda5e-a4be-4bd0-99c2-478ef193e57a service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:48:03.596058 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1465/2936] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:48:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:48:04.791555 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600893dc-0408-43f6-ad7a-181b1b207262 service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:48:04.800343 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600893dc-0408-43f6-ad7a-181b1b207262 service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:48:04.804171 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd431520-058b-4093-8cf6-ad404dedc72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:48:04.804878 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd431520-058b-4093-8cf6-ad404dedc72f service neutron] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:48:04.805157 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd431520-058b-4093-8cf6-ad404dedc72f service neutron] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:48:04.805424 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd431520-058b-4093-8cf6-ad404dedc72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:48:04.816045 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd431520-058b-4093-8cf6-ad404dedc72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:48:04.817472 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd431520-058b-4093-8cf6-ad404dedc72f service neutron] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:48:04.829292 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1472/2937] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:48:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:48:13.590710 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=12 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:48:14.823984 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=12 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:48:16.098205 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018eda5e-a4be-4bd0-99c2-478ef193e57a service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:48:16.107797 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018eda5e-a4be-4bd0-99c2-478ef193e57a service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:48:16.112864 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fc805-d643-4129-8144-790b34728f2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa65b6854dc54e289cc3e8e0b3746dd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:48:16.113482 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc805-d643-4129-8144-790b34728f2d service neutron] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:48:16.113761 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc805-d643-4129-8144-790b34728f2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:48:16.114026 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc805-d643-4129-8144-790b34728f2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:48:16.182473 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fc805-d643-4129-8144-790b34728f2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:48:16.183940 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fc805-d643-4129-8144-790b34728f2d service neutron] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:48:16.193694 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1466/2938] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:48:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:48:19.076369 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd431520-058b-4093-8cf6-ad404dedc72f service neutron] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:48:19.138814 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd431520-058b-4093-8cf6-ad404dedc72f service neutron] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:48:19.147817 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4f706-5231-450b-b4ea-9ac2363367d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784b6147de91435292e4cf3940bcad9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:48:19.148737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4f706-5231-450b-b4ea-9ac2363367d7 service placement] REQUEST_METHOD: `GET` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:48:19.148737 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4f706-5231-450b-b4ea-9ac2363367d7 service placement] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:48:19.148915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4f706-5231-450b-b4ea-9ac2363367d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:48:19.150893 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-f4f4f706-5231-450b-b4ea-9ac2363367d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:48:19.189022 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4f706-5231-450b-b4ea-9ac2363367d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:48:19.190936 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4f706-5231-450b-b4ea-9ac2363367d7 service placement] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:48:19.203070 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1473/2939] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:48:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:48:22.325090 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fc805-d643-4129-8144-790b34728f2d service neutron] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:48:22.378276 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fc805-d643-4129-8144-790b34728f2d service neutron] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:48:22.383021 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e072f-5e26-436e-8fd3-93e7f552d52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:48:22.383662 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e072f-5e26-436e-8fd3-93e7f552d52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:48:22.383941 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e072f-5e26-436e-8fd3-93e7f552d52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:48:22.384214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e072f-5e26-436e-8fd3-93e7f552d52c admin admin] PATH_INFO: `/v3/users/69ec3d4f006c4f8f815b0cad9f933208` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:48:22.385367 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e072f-5e26-436e-8fd3-93e7f552d52c admin admin] RBAC: Authorizing `identity:delete_user(user_id=69ec3d4f006c4f8f815b0cad9f933208)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:48:22.387379 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e072f-5e26-436e-8fd3-93e7f552d52c admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:48:22.440193 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1f3e072f-5e26-436e-8fd3-93e7f552d52c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69ec3d4f006c4f8f815b0cad9f933208'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:22.440648 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1f3e072f-5e26-436e-8fd3-93e7f552d52c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69ec3d4f006c4f8f815b0cad9f933208'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:22.441244 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1f3e072f-5e26-436e-8fd3-93e7f552d52c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69ec3d4f006c4f8f815b0cad9f933208'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:22.448817 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1f3e072f-5e26-436e-8fd3-93e7f552d52c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69ec3d4f006c4f8f815b0cad9f933208'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:22.460603 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-1f3e072f-5e26-436e-8fd3-93e7f552d52c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69ec3d4f006c4f8f815b0cad9f933208'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:22.474912 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1467/2940] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:48:22 2022] DELETE /identity/v3/users/69ec3d4f006c4f8f815b0cad9f933208 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:48:22.657761 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4f706-5231-450b-b4ea-9ac2363367d7 service placement] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:48:22.668786 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4f706-5231-450b-b4ea-9ac2363367d7 service placement] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:48:22.733379 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:48:22.733967 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:48:22.734243 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:48:22.734526 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] PATH_INFO: `/v3/projects/5c52563210cf435696f3e3ae3907b0fb` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:48:22.735717 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c52563210cf435696f3e3ae3907b0fb)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:48:22.737642 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:48:22.788631 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c52563210cf435696f3e3ae3907b0fb', 'request_id': 'req-7261dd82-dcc6-4e26-9038-4eeae9535262'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:22.794982 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c52563210cf435696f3e3ae3907b0fb', 'request_id': 'req-7261dd82-dcc6-4e26-9038-4eeae9535262'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:22.800995 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c52563210cf435696f3e3ae3907b0fb', 'request_id': 'req-7261dd82-dcc6-4e26-9038-4eeae9535262'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:22.829699 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] The token cache is being invalidate because project 5c52563210cf435696f3e3ae3907b0fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:48:22.829990 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:22.830764 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:22.832403 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1474/2941] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:48:22 2022] DELETE /identity/v3/projects/5c52563210cf435696f3e3ae3907b0fb => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:48:22.837272 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e072f-5e26-436e-8fd3-93e7f552d52c admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:48:22.839028 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1f3e072f-5e26-436e-8fd3-93e7f552d52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:48:22.852520 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e072f-5e26-436e-8fd3-93e7f552d52c admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:48:22.877931 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:48:22.913945 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fbc37-bf35-4621-9fcb-5f89ed4ec8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:48:22.914532 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fbc37-bf35-4621-9fcb-5f89ed4ec8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:48:22.914822 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fbc37-bf35-4621-9fcb-5f89ed4ec8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:48:22.915099 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fbc37-bf35-4621-9fcb-5f89ed4ec8be admin admin] PATH_INFO: `/v3/users/5b384779760949de996835ac355e5179` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:48:22.916263 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fbc37-bf35-4621-9fcb-5f89ed4ec8be admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b384779760949de996835ac355e5179)` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:48:22.918214 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fbc37-bf35-4621-9fcb-5f89ed4ec8be admin admin] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:48:22.969164 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ff1fbc37-bf35-4621-9fcb-5f89ed4ec8be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b384779760949de996835ac355e5179'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:22.969877 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ff1fbc37-bf35-4621-9fcb-5f89ed4ec8be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b384779760949de996835ac355e5179'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:22.970575 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ff1fbc37-bf35-4621-9fcb-5f89ed4ec8be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b384779760949de996835ac355e5179'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:22.977508 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ff1fbc37-bf35-4621-9fcb-5f89ed4ec8be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b384779760949de996835ac355e5179'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:22.989937 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.notifications [None req-ff1fbc37-bf35-4621-9fcb-5f89ed4ec8be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b384779760949de996835ac355e5179'} {{(pid=81510) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:23.002753 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1468/2942] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Sep 9 16:48:22 2022] DELETE /identity/v3/users/5b384779760949de996835ac355e5179 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:48:23.436097 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] Authenticating user token {{(pid=81511) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:48:23.437583 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.fernet_utils [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81511) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:48:23.450523 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261dd82-dcc6-4e26-9038-4eeae9535262 admin admin] Validating token access rules against request {{(pid=81511) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:48:23.464668 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:48:23.510338 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b546a-a272-489d-878a-5cd4b022e61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cbbd1503324af1af6df581aea570b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5e628b418b44cb0b8fcf5460ded8ac8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81511) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:48:23.511018 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b546a-a272-489d-878a-5cd4b022e61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:48:23.511294 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b546a-a272-489d-878a-5cd4b022e61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:48:23.511563 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b546a-a272-489d-878a-5cd4b022e61e admin admin] PATH_INFO: `/v3/projects/902c3f9c73124ba3af7787d3a20ed898` {{(pid=81511) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:48:23.512735 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b546a-a272-489d-878a-5cd4b022e61e admin admin] RBAC: Authorizing `identity:delete_project(project_id=902c3f9c73124ba3af7787d3a20ed898)` {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:48:23.514701 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b546a-a272-489d-878a-5cd4b022e61e admin admin] RBAC: Authorization granted {{(pid=81511) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:48:23.561929 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-611b546a-a272-489d-878a-5cd4b022e61e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '902c3f9c73124ba3af7787d3a20ed898', 'request_id': 'req-611b546a-a272-489d-878a-5cd4b022e61e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:23.567887 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-611b546a-a272-489d-878a-5cd4b022e61e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '902c3f9c73124ba3af7787d3a20ed898', 'request_id': 'req-611b546a-a272-489d-878a-5cd4b022e61e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:23.573915 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-611b546a-a272-489d-878a-5cd4b022e61e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '902c3f9c73124ba3af7787d3a20ed898', 'request_id': 'req-611b546a-a272-489d-878a-5cd4b022e61e'} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:23.587522 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-611b546a-a272-489d-878a-5cd4b022e61e admin admin] The token cache is being invalidate because project 902c3f9c73124ba3af7787d3a20ed898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81511) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 16:48:23.587802 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-611b546a-a272-489d-878a-5cd4b022e61e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:23.588408 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG keystone.notifications [None req-611b546a-a272-489d-878a-5cd4b022e61e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81511) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 16:48:23.589970 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: [pid: 81511|app: 0|req: 1475/2943] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Sep 9 16:48:23 2022] DELETE /identity/v3/projects/902c3f9c73124ba3af7787d3a20ed898 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 16:48:32.999794 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=41,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:48:33.584891 dv-cinder-831247-27-iscsi devstack@keystone.service[81511]: DEBUG dbcounter [-] [81511] Writing DB stats keystone:SELECT=43,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81511) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 16:50:08.336725 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fbc37-bf35-4621-9fcb-5f89ed4ec8be admin admin] Authenticating user token {{(pid=81510) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 16:50:08.338342 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-ff1fbc37-bf35-4621-9fcb-5f89ed4ec8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:50:08.432793 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fbc37-bf35-4621-9fcb-5f89ed4ec8be admin admin] Validating token access rules against request {{(pid=81510) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 16:50:08.494412 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970a8fa-14c1-4e66-a22c-9b12651638b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784b6147de91435292e4cf3940bcad9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '253ff30a56444df28d44c4b7eaadb276', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81510) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 16:50:08.495050 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970a8fa-14c1-4e66-a22c-9b12651638b5 service placement] REQUEST_METHOD: `GET` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 16:50:08.495330 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970a8fa-14c1-4e66-a22c-9b12651638b5 service placement] SCRIPT_NAME: `/identity` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 16:50:08.495599 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970a8fa-14c1-4e66-a22c-9b12651638b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81510) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 16:50:08.497497 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.fernet_utils [None req-1970a8fa-14c1-4e66-a22c-9b12651638b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81510) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 16:50:08.509572 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970a8fa-14c1-4e66-a22c-9b12651638b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 16:50:08.512068 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970a8fa-14c1-4e66-a22c-9b12651638b5 service placement] RBAC: Authorization granted {{(pid=81510) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 16:50:08.524120 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: [pid: 81510|app: 0|req: 1469/2944] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Sep 9 16:50:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 16:50:18.519830 dv-cinder-831247-27-iscsi devstack@keystone.service[81510]: DEBUG dbcounter [-] [81510] Writing DB stats keystone:SELECT=67 {{(pid=81510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}